Enhancing E-commerce Search with LLM-Powered Semantic Retrieval

picnic · E-commerce · 2024 · source Agentic: no · promotional bias: med · summary-only (no fetched source) · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Grocery e-commerce semantic search

Picnic needed search retrieval that could handle multilingual, typo-prone, ambiguous grocery queries across a large mobile catalog without adding unacceptable LLM latency to search-as-you-type experiences.

Constraints:

• The fetched source page is not the case-study article, so this extraction uses the ZenML summary only.
• Picnic serves customers across the Netherlands, Germany, and France.
• Search must work across millions of unique terms, multiple languages, typos, and ambiguous intent.
• Real-time LLM calls would add seconds of latency, which is unacceptable for search.
• OpenSearch retrieval imposes a practical 1536-dimensional embedding limit.

Architecture

Picnic used LLMs as an offline semantic-search enhancer rather than a live generator. GPT-3.5-turbo creates contextual product or search-term descriptions, OpenAI text-embedding-3-small embeds those descriptions into 1536-dimensional vectors compatible with OpenSearch, common search terms and product/recipe content are precomputed, and caching plus sanity checks keep the production search path fast and reliable.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  historical-search-terms["Historical search terms"] --> gpt-description-generator["GPT description generator"]
  gpt-description-generator --> embedding-generator["Embedding generator"]
  embedding-generator --> opensearch-indexes["OpenSearch indexes"]
  cache-and-sanity-checks["Cache and sanity checks"] --> live-search-retriever["Live search retriever"]
  opensearch-indexes --> live-search-retriever
  live-search-retriever --> ab-testing-and-monitoring["AB testing and monitoring"]
  ab-testing-and-monitoring --> historical-search-terms

Components

Data flow

Historical search data identifies common search terms. GPT-generated descriptions and embeddings are precomputed for search terms and catalog content, stored in OpenSearch indexes, and then live queries are matched against precomputed search-term embeddings before semantic retrieval of products and recipes.

Why this architecture

Picnic chose precomputed semantic retrieval because it wanted LLM-level query understanding without the seconds-long latency of synchronous LLM calls. GPT-3.5-turbo was selected over GPT-4-turbo for comparable performance with lower latency, and text-embedding-3-small matched OpenSearch's dimensionality constraint.

Models used: GPT-3.5-turbo · GPT-4-turbo · text-embedding-3-small · OpenSearch

Scale

• Picnic operates across the Netherlands, Germany, and France.
  “Picnic is a European grocery delivery company operating across the Netherlands, Germany, and France”summary, stated
• The search system handles millions of unique search terms.
  “With millions of unique search terms being used by customers, developing a search system that delivers accurate, fast results became a prime candidate for LLM-enhanced solutions.”summary, stated
• Precomputation covers approximately 99% of search terms customers use.
  “they can identify and precompute embeddings for approximately 99% of search terms that customers use.”summary, stated
• Embeddings are 1536-dimensional vectors.
  “text-embedding-3-small model, which produces 1536-dimensional vectors.”summary, stated
• Precomputation lets the system deliver results in milliseconds instead of LLM-response seconds.
  “allowing the system to deliver results in milliseconds rather than the seconds that typical LLM responses require.”summary, stated

Alternatives considered (roads not taken)

Call LLM APIs at query time.

Rejected because: Synchronous LLM calls would introduce unacceptable latency for search-as-you-type.

Use GPT-4-turbo for prompt generation.

Rejected because: GPT-3.5-turbo performed comparably while being significantly faster.

Use the largest embedding model available.

Rejected because: OpenSearch's 1536-dimensional limit made text-embedding-3-small the practical fit.

Patterns

semantic-caching (serving)

Picnic precomputes and caches embeddings for common search terms to avoid live LLM calls.

rag-retrieval (retrieval)

OpenSearch retrieves semantically relevant products and recipes from vector embeddings.

latency-optimization (serving)

The architecture avoids synchronous LLM latency by doing expensive generation and embedding work upfront.

eval-driven-development (evaluation)

Offline optimization is followed by controlled A/B testing for conversion, click-through, and satisfaction.

Tradeoffs

LLM placement

Chose Precompute embeddings and descriptions offline. over Synchronous LLM inference in live search..

Rationale: The precomputed approach preserves millisecond search latency.

Model selection

Chose GPT-3.5-turbo for prompt generation. over GPT-4-turbo..

Rationale: GPT-3.5-turbo was faster while performing comparably for the task.

Embedding model

Chose text-embedding-3-small. over A larger embedding model exceeding OpenSearch's efficient retrieval limit..

Rationale: It outputs 1536-dimensional vectors aligned with OpenSearch's maximum dimensionality constraint.

Gotchas (what bit them)

The case study does not disclose concrete search-lift numbers.

Root cause: It names conversion, click-through, and satisfaction goals without publishing before/after results.

Resolution: Treat reported evaluation as process evidence rather than quantified impact evidence.

Real-time LLM calls would break search latency.

Root cause: Typical LLM responses take seconds, while search needs millisecond-level response.

Resolution: Precompute common search-term and catalog embeddings.

Model and embedding outputs can drift over time.

Root cause: Language models vary with updates and model iterations.

Resolution: Use sanity checks for embedding consistency and expected length.

Offline evaluation can optimize against imperfect historical ground truth.

Root cause: Past search results may not represent optimal future outcomes.

Resolution: Use offline evaluation for initial tuning and online A/B testing for real user validation.

Evaluation

Picnic used offline optimization for search parameters, prompts, dimensions, and models, then validated with A/B testing on real conversion, click-through, and customer satisfaction.

Metrics: Conversion rates · Click-through rates · Customer satisfaction · Embedding consistency and length · Latency and system stability

Outcomes

• Enabled semantic matching for multilingual and ambiguous grocery search terms.
• Covered approximately 99% of customer search terms through precomputation.
• Kept live retrieval in the millisecond range instead of LLM-response seconds.
• Created an evaluation path from offline optimization to real-user A/B testing.

Picnic operates in three countries, handles millions of search terms, precomputes roughly 99% of customer terms, and uses 1536-dimensional embeddings.