Enhancing Vector Similarity Search with LLM-Based Reranking

grab · Tech · 2024 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Vector search reranking

Grab needed search results that preserve vector search efficiency while handling nuanced queries involving negation, constraints, or conceptual intent that raw similarity search can mis-rank.

Constraints:

• Raw vector search can treat opposite statements such as liking and not liking fishing as close.
• LLM reranking adds latency and must be considered for real applications.
• The experiment used small synthetic datasets and limited query sets.
• The method is most useful when complex context matters; raw vector search remains efficient for straightforward queries.
• Structured-data retrieval was the primary focus.

Architecture

Grab proposed a two-step retrieval architecture. FAISS and OpenAI embeddings perform efficient vector similarity search to shortlist candidate entities, then GPT-4o reranks the shortlist using the original query and descriptive instructions to select the most contextually relevant results. This keeps vector search as the fast narrowing stage and uses the LLM only for nuanced ranking.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  original-query["Original query"] --> embedding-generator["Embedding generator"]
  embedding-generator --> faiss-vector-search["FAISS vector search"]
  faiss-vector-search --> llm-reranker["LLM reranker"]
  llm-reranker --> final-search-results["Final search results"]
  final-search-results --> experiment-evaluation["Experiment evaluation"]
  experiment-evaluation --> original-query

Components

Data flow

The query is embedded and sent to FAISS for an initial shortlist of candidates. The shortlist and original query are then passed to GPT-4o, which reasons over context, negation, and constraints to rerank the candidates and return the top matches.

Why this architecture

Grab used vector search for the efficient shortlist and LLM reranking for nuanced contextual understanding, giving the system a way to handle negation and conceptual requirements without asking the LLM to search the full dataset.

Models used: FAISS · text-embedding-ada-002 · GPT-4o

Scale

• The synthetic food dataset contained 100 dishes.
  “A collection of 100 dishes with their titles and descriptions.”article, stated
• The synthetic tourist dataset contained 100 tourist spots.
  “A collection of 100 tourist spots in Asia, including their names, cities, countries, and descriptions.”article, stated
• The proposed experiment shortlisted 15 entities before LLM reranking.
  “we first get a shortlist of 15 entity matches from FAISS for the same query”article, stated
• Grab observed similar internal relevance improvements on 4500+ relational-database rows.
  “we have witnessed similar enhancement in search result relevance when we deployed this solution internally within Grab for larger datasets, for example, 4500+ rows of data stored in a relational database.”article, stated
• Grab operates across 700 cities in eight countries.
  “Grab offers a wide range of on-demand services in the region, including mobility, food, package and grocery delivery services, mobile payments, and financial services across 700 cities in eight countries.”article, stated

Alternatives considered (roads not taken)

Use raw vector similarity search only.

Rejected because: Raw vector search struggled on negation and nuanced conceptual constraints.

Use LLM-assisted reranking for every query regardless of complexity.

Rejected because: Raw similarity search remains viable for simple queries when computational efficiency matters.

Patterns

retrieval-ranking (retrieval)

GPT-4o reranks a FAISS shortlist by contextual relevance to the original query.

rag-retrieval (retrieval)

FAISS and embeddings perform the initial vector similarity shortlist.

cost-aware-routing (serving)

The architecture reserves the LLM for reranking a shortlist rather than scanning the full dataset.

Tradeoffs

Retrieval architecture

Chose Vector shortlist plus LLM reranking. over Raw vector search only..

Rationale: The two-stage design keeps fast shortlisting while adding natural-language understanding for complex constraints.

Shortlist size

Chose Feed 15 candidates to the LLM in the experiment. over Reranking the whole dataset or only the raw top three..

Rationale: The shortlist gives the LLM enough candidates while bounding cost and context.

Gotchas (what bit them)

Vector similarity can confuse negation.

Root cause: Semantically adjacent phrases with opposite meanings may be close in vector space.

Resolution: Use an LLM reranker to reason over the original query and candidate context.

The experimental evidence is limited.

Root cause: The published experiment used small synthetic datasets and limited queries.

Resolution: Evaluate on larger and more diverse datasets before broad production use.

LLM reranking can add production latency.

Root cause: The architecture introduces an additional LLM query after vector search.

Resolution: Consider latency implications and reserve the approach for cases where added context is worth it.

Evaluation

Grab compared raw vector search top-three results with LLM-assisted reranked top-three results on two GPT-4o-generated synthetic datasets and also notes internal deployment on 4500+ relational rows.

Metrics: Top-three relevance · Complex-query relevance · Negation handling · Structured dataset size: 100 rows each · Internal dataset size: 4500+ rows

Outcomes

• LLM reranking improved complex-query relevance over raw vector search.
• Raw vector search remained adequate for straightforward keyword-aligned queries.
• Internal Grab deployment on 4500+ database rows showed similar relevance enhancement.
• The team identified shortlist size and latency as future production considerations.

Published experiments used two 100-row synthetic datasets and a 15-candidate shortlist; internal use reached 4500+ rows.