Enhancing E-commerce Search with GPT-based Query Expansion

whatnot · E-commerce · 2023 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: E-commerce search query expansion

Whatnot needed to handle misspellings and abbreviations in marketplace search so users would not see empty or low-quality results for queries that should map to existing inventory.

Constraints:

Search is latency-sensitive and should ideally remain under 250ms.
Misspellings such as jewlery can make users believe the marketplace lacks content.
Acronyms such as lv and nyfw need expansion into brand or event names.
Real-time LLM inference is too slow and costly for every search request.
Token-specific expansion does not automatically support reverse matching from full phrase to abbreviation.

Architecture

Whatnot built an offline GPT-based query expansion pipeline. Search logs are tokenized and normalized, frequent tokens from the last 14 days are sent to GPT on an ad hoc or scheduled basis, GPT returns spelling corrections and abbreviation expansions with confidence, and the outputs are stored in a production key-value cache. At request time, the search service tokenizes the query, looks up cached expansions, augments the query S-expression, and generates results from the original and expanded query terms.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  search-event-logs["Search event logs"] --> tokenization-job["Tokenization job"]
  tokenization-job --> offline-gpt-rectification["Offline GPT rectification"]
  offline-gpt-rectification --> query-expansion-cache["Query expansion cache"]
  query-expansion-cache --> runtime-query-lookup["Runtime query lookup"]
  runtime-query-lookup --> search-result-generation["Search result generation"]
  search-result-generation --> search-event-logs

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Search event logs`	Records queries, filters, SERP tabs, and joined behavior across search sessions.	`Tokenization job`
`Tokenization job`	Normalizes queries and extracts frequently occurring unigram tokens.	`Offline GPT rectification`
`Offline GPT rectification`	Detects misspellings and expands acronyms or abbreviations outside the live request path.	`Query expansion cache`
`Query expansion cache`	Production key-value store mapping original tokens to corrections, expansions, and confidence levels.	`Runtime query lookup`
`Runtime query lookup`	Tokenizes live user queries and retrieves cached expansions.	`Search result generation`
`Search result generation`	Combines original and expanded queries to return products, shows, users, or other SERP results.	`Search event logs`

Data flow

Search event logs produce normalized query tokens. Frequently used tokens are processed offline by GPT to generate corrections and expansions, which are stored in a query-expansion cache. During live search, query tokens are looked up in the cache and used to augment the search expression before generating the result page.

“As illustrated in the flowchart above, our offline query expansion generation process follows these steps.”article, stated

“At request time, when a user executes a query during search time, our process follows these steps:”article, stated

Why this architecture

Whatnot used GPT offline because it improved misspelling and abbreviation handling without adding LLM latency to the live search path. The key-value cache converts expensive language-model reasoning into low-latency lookup at query time.

“This GPT call is made on an ad hoc/scheduled basis outside of the production code path since the user value of Search is heavily predicated on low latency (ideally sub-250ms).”article, stated

“This is a tier in a production-level key-value store that maps from original query tokens to the lists of potential corrections/expansions, along with their associated confidence levels.”article, stated

Models used: GPT

Scale

Frequent tokens are gathered from the past 14 days of search usage.
“We gather frequently occurring tokens by summarizing their usage over the past 14 days.”article, stated
Tokens used more than three times are included in the GPT process.
“if a token has been utilized in search queries more than 3 times during this period, we consider it as a token to be included in the subsequent GPT process.”article, stated
The live search experience targets sub-250ms latency.
“the user value of Search is heavily predicated on low latency (ideally sub-250ms).”article, stated
The GPT-based method reduced irrelevant content by more than 50% for misspelling and abbreviation queries.
“we reduced irrelevant content by more than 50% compared to our previous method”article, stated

Alternatives considered (roads not taken)

Call GPT in the live search request path.

Rejected because: Live search value depends on low latency, ideally under 250ms.

“outside of the production code path since the user value of Search is heavily predicated on low latency”article, stated

Continue the previous query expansion method.

Rejected because: GPT rectification reduced irrelevant content by more than 50%.

“Compared to our previous query expansion method, this new GPT rectification-based approach has yielded substantial improvements”article, stated

Build and maintain a knowledge graph for brand/entity misspellings.

Rejected because: GPT already knew many real-world entities and provided basic handling without knowledge-graph maintenance.

“This ability to handle real-world entities well means that we can do reasonable, basic handling of these cases in Search”article, stated

Patterns

`query-rewriting` (retrieval)

GPT generates spelling corrections and abbreviation expansions for search tokens.

“The GPT model then generates corresponding spelling corrections and abbreviation expansions.”article, stated

`semantic-caching` (serving)

Generated expansions are cached in a production key-value store for low-latency lookup at search time.

“After receiving outputs from the GPT model, we put them into our query expansion cache.”article, stated

`batch-offline-processing` (operations)

The GPT rectification call runs ad hoc or on a schedule outside the production request path.

“This GPT call is made on an ad hoc/scheduled basis outside of the production code path”article, stated

`latency-optimization` (serving)

LLM work is shifted offline so the live search path remains compatible with sub-250ms expectations.

“Search is heavily predicated on low latency (ideally sub-250ms).”article, stated

Tradeoffs

LLM placement

Chose Offline GPT generation plus runtime cache lookup. over Real-time GPT inference for each user query..

Rationale: This preserves low search latency while still benefiting from GPT knowledge.

“outside of the production code path since the user value of Search is heavily predicated on low latency”article, stated

Expansion granularity

Chose Frequently occurring tokens and unigrams. over Full n-gram bidirectional expansion in the initial system..

Rationale: Token-level generation was simpler to operationalize but left reverse phrase-to-acronym expansion for future work.

“To support this, we will need to either 1) apply the equivalent query expansion process at indexing time 2) perform GPT rectification upon ngrams.”article, stated

Gotchas (what bit them)

Misspellings created false empty-search impressions.

Root cause: Users naturally assumed Whatnot lacked content when a misspelled query returned sparse results.

Resolution: Use GPT-generated spelling corrections in query expansion.

“most users naturally assume that Whatnot lacks jewelry-related content when they encounter a nearly empty “jewlery” search results page.”article, stated

Acronym and abbreviation queries had lower engagement.

Root cause: Queries such as lv and nyfw did not automatically expand to their intended full phrases.

Resolution: Generate abbreviation expansions and use confidence-weighted cache lookups at request time.

“acronym/abbreviation queries, such as “lv” for “louis vuitton” or “nyfw” for “new york fashion week” tended to result in a low count of results/lower downstream engagement rates.”article, stated

Token-specific expansion was one-directional.

Root cause: The system expanded sdcc to San Diego Comic Con, but not necessarily the reverse.

Resolution: The team identified indexing-time expansion or n-gram GPT rectification as next steps.

“our current token-specific approach means that a user searching for “san diego comic con” will not get results matching “sdcc”.”article, stated

Evaluation

Whatnot compared the GPT rectification-based approach against the previous query expansion method for misspelling and abbreviation queries, measuring irrelevant content reduction.

Metrics: More than 50% reduction in irrelevant content · Sub-250ms live search latency requirement · Frequency threshold: more than 3 token uses in 14 days

“For queries containing misspellings or abbreviations, we reduced irrelevant content by more than 50% compared to our previous method.”article, stated

Outcomes

Reduced irrelevant content by more than 50% for misspelling and abbreviation queries.
Improved search handling for examples such as jewlery, lv, nyfw, and sdcc.
Kept LLM inference out of the production request path through scheduled offline generation and cache lookup.
Simplified expansion generation and serving compared with the previous method.

Whatnot processes frequent search tokens over a 14-day window, only includes tokens used more than three times, and protects a sub-250ms live search target.

“this new GPT rectification-based approach has yielded substantial improvements in query expansion accuracy while also streamlining the generation and serving process significantly.”article, stated