LLM-Powered Search Relevance Re-Ranking System

leboncoin · E-commerce · 2023 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Marketplace search relevance

LeBonCoin needed to improve search-result ordering across a large, volatile user-generated classifieds catalog while preserving the low latency expected from a production search engine.

Constraints:

• The marketplace serves nearly 30 million monthly active users and more than 60 million classified ads.
• Ads are user-generated and described in each user's own words, making lexical search alone brittle.
• Search serving has only a few dozen milliseconds of latency budget and peak throughput of thousands of requests per second.
• A neural model cannot afford to rescore the entire result set for every query.
• Initial retrieval quality still constrains what the re-ranker can improve.

Architecture

LeBonCoin deployed a DistilBERT-based bi-encoder re-ranker beside its existing ElasticSearch search stack. The ad encoder precomputes embeddings for catalog ads into a vector database, while the query encoder and scorer run at request time on the top-k ads returned by ElasticSearch. The neural scores are combined with the original ElasticSearch scores, then the re-ranked top-k results are shown ahead of the residual result set.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  Dataset["Click-model dataset builder"] --> Ad["Ad encoder"]
  Dataset --> Query["Query encoder"]
  Ad --> Vector["Vector database"]
  Elastic["ElasticSearch retriever"] --> Scorer["Scorer"]
  Vector --> Scorer
  Query --> Scorer
  Scorer --> Final["Final ranker"]
  Elastic --> Final

Components

Data flow

The system builds training data from click-model signals, trains a two-tower ad/query encoder with a scorer, embeds the ad catalog offline through the ad encoder, retrieves an initial pool with ElasticSearch at query time, fetches top-k ad vectors, computes the query embedding and click-propensity scores, combines neural and ElasticSearch scores, and displays the re-ranked top-k ahead of unre-ranked residual ads.

Why this architecture

LeBonCoin chose a bi-encoder and top-k re-ranking architecture because full neural scoring of all ads at request time would not fit search latency and throughput constraints. Offline ad embedding moves the expensive catalog-side work out of the request path, while ElasticSearch preserves high-recall first-stage retrieval and the neural scorer improves ordering where users look first.

Models used: DistilBERT · bi-encoder · custom MLP layers

Scale

• LeBonCoin serves almost 30 million unique monthly active users.
  “With almost 30 millions unique monthly active users and more than 60 millions different classified ads in our catalogue”article, stated
• The catalog contains more than 60 million classified ads.
  “With almost 30 millions unique monthly active users and more than 60 millions different classified ads in our catalogue”article, stated
• The online serving budget is only a few dozen milliseconds.
  “within low latencies time window (few dozens of milliseconds allowed) while facing high throughput”article, stated
• Peak throughput reaches up to a few thousand requests per second.
  “while facing high throughput (up to few thousands of requests per second at peak time).”article, stated

Alternatives considered (roads not taken)

Rely only on ElasticSearch ranking

Rejected because: ElasticSearch remains the retriever, but LeBonCoin added the neural re-ranker because the goal was to improve the ordering of ads for a query.

Compute all ad representations in the online request path

Rejected because: The catalog-side computation can be done before serving, avoiding unnecessary load on the real-time path.

Re-rank the entire retrieved result set

Rejected because: The production design limits neural scoring to the top-k candidates and appends residual ads afterward.

Patterns

multi-stage-retrieval (retrieval)

ElasticSearch performs first-stage retrieval, and the neural re-ranker only rescored top-k candidates before final ranking.

latency-aware-model-routing (serving)

The architecture routes expensive ad encoding to offline batch computation and keeps only query encoding and scoring in the online path.

eval-driven-development (evaluation)

LeBonCoin evaluated the re-ranker through business and user-experience ranking metrics before presenting it as the first successful production iteration.

Tradeoffs

Use a bi-encoder with offline ad embeddings

Chose separate ad and query encoders with precomputed ad vectors over jointly processing every query-ad pair online.

Rationale: The chosen architecture fits the latency and throughput constraints by moving catalog-side computation out of the request path.

Use staged re-ranking rather than neural full-corpus ranking

Chose ElasticSearch retrieval plus top-k neural re-ranking over running the neural scorer over all retrieved or catalog ads.

Rationale: The neural stage focuses compute on candidates most likely to affect the visible result list while preserving a complete listing.

Gotchas (what bit them)

Initial retrieval bounds what the neural re-ranker can improve.

Root cause: Only top-k candidates from ElasticSearch are sent to the re-ranker, so missed candidates remain outside the neural stage.

Resolution: Use ElasticSearch as a recall-focused retriever and reserve neural scoring for the most visible candidates.

Real-time neural ranking can overload a search serving path.

Root cause: Deep neural networks must serve within dozens of milliseconds while handling thousands of requests per second.

Resolution: Precompute ad embeddings offline and run only query-side encoding plus scoring online.

Click-derived training labels inherit search-position and exposure bias.

Root cause: The dataset relies on implicit user feedback rather than explicit unbiased relevance judgments.

Resolution: Use statistical filtering and example weighting when building the contrastive learning dataset.

Evaluation

LeBonCoin assessed the first production re-ranker iteration through business targets and search quality KPIs after deployment.

Metrics: click rate · contact rate · nDCG · average clicked position · average contacted position

Outcomes

• The re-ranker improved click and contact rates by up to 5%.
• User-experience KPIs including nDCG and clicked/contacted positions improved by up to 10%.
• The deployment demonstrated production viability for neural search ranking under marketplace latency and throughput constraints.

The deployed search architecture served nearly 30 million monthly active users and more than 60 million ads under peak loads of up to a few thousand requests per second.