Building Price Prediction and Similar Item Search Models for E-commerce

ebay · E-commerce · 2024 · source Agentic: no · promotional bias: low · summary-only (no fetched source) · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: E-commerce price guidance and similar-item retrieval

eBay needed to help sellers price second-hand and niche products, especially sports trading cards, where ordinary keyword search fails because titles contain abbreviations, slang, domain terminology, item condition, quantity, player, year, rarity, and grading details that strongly affect value.

Constraints:

• The system must operate over historically sold items at eBay scale.
• Sports-card pricing depends on subtle attributes such as autograph status, rarity, condition, grading, quantity, and player/year combinations.
• Traditional text search misses domain equivalences such as RC/Rookie Card and signed/auto.
• Pure price accuracy can undermine seller trust if the comparison items look semantically unrelated.
• Business stakeholders need a controllable trade-off between retrieval trust and pricing accuracy.

Architecture

eBay built an embedding-based retrieval and price-guidance system rather than a generative LLM agent. Product titles are encoded with a custom BERT-style encoder into dense vectors. Historically sold items are stored in a vector database, and new seller listings run k-nearest-neighbor search to retrieve comparable sold items for pricing guidance. Training uses contrastive pairs mined from user search sessions and clicks, in-batch negatives, hard negatives based on similar-looking but different-priced items, and a multi-task architecture that alternates semantic-similarity and price-prediction objectives. A tunable alpha controls the position on the semantic-trust versus price-accuracy curve.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  seller-listing-form["Seller listing form"] --> title-embedding-service["Title embedding service"]
  historical-sold-listings["Historical sold listings"] --> vector-database["Vector database"]
  title-embedding-service --> vector-database
  vector-database --> knn-retrieval-service["KNN retrieval service"]
  knn-retrieval-service --> price-guidance-experience["Price guidance experience"]
  user-behavior-logs["User behavior logs"] --> training-pair-generator["Training-pair generator"]
  historical-sold-listings --> training-pair-generator
  training-pair-generator --> multi-task-training-pipeline["Multi-task training pipeline"]
  negative-sampler["Negative sampler"] --> multi-task-training-pipeline
  multi-task-training-pipeline --> title-embedding-service
  evaluation-set["Evaluation set"] --> trade-off-selector["Trade-off selector"]
  trade-off-selector --> title-embedding-service
  multi-task-training-pipeline --> trade-off-selector

Components

Data flow

The pipeline starts with historical sold listings and user behavior data. Search sessions with specific queries and multiple clicked items generate positive item pairs; batch construction supplies in-batch negatives; hard negative mining finds items that appear similar but differ on price-relevant attributes. A BERT encoder is trained on semantic similarity and price prediction objectives. At serving time, a seller's listing title is embedded, KNN search retrieves similar sold items from the vector database, and the product experience can show comparable sales or a recommended price.

Why this architecture

Dense embeddings were chosen because the domain contains abbreviations and semantic equivalences that keyword search cannot reliably match. Custom BERT training was preferred over off-the-shelf embeddings because eBay had domain data and compute. Multi-task learning was added because the team found that optimizing only semantic similarity preserved trust but hurt price accuracy, while optimizing only price improved MAE but could retrieve semantically confusing comparisons.

Models used: BERT-based transformer encoder · Siamese semantic similarity model · Title-to-price prediction transformer · Multi-task shared-weight model

Scale

• eBay operates at approximately 2 billion listings.
  “eBay operates at massive scale with approximately 2 billion listings”summary, stated
• The marketplace has over 130 million active buyers.
  “over 130 million active buyers”summary, stated
• The marketplace spans 190 different marketplace sites worldwide.
  “190 different marketplace sites worldwide”summary, stated
• The negative sampling setup uses batch size 64, yielding 63 implicit negatives per positive pair.
  “With batch sizes of 64, each positive pair gets 63 implicit negative examples”summary, stated
• Title-to-price embeddings reached MAE 29 versus MAE 38 for semantic similarity.
  “Title-to-Price approach: MAE of $29”summary, stated

  “Semantic Similarity approach: MAE of $38”summary, stated
• The production embedding model must encode millions of historical listings.
  “The embedding model must efficiently encode millions of historical listings”summary, stated

Alternatives considered (roads not taken)

Traditional keyword search over listing titles.

Rejected because: It cannot handle sports-card abbreviations, slang, and domain-specific equivalences such as RC/Rookie Card or signed/auto.

Off-the-shelf general embedding models.

Rejected because: Domain-specific models trained on eBay data performed substantially better when training data and GPU resources were available.

Soft random negatives only.

Rejected because: Random negatives did not teach pricing-relevant fine-grained distinctions, so hard negatives were mined from similar-looking but differently priced cards.

Pure semantic-similarity Siamese training.

Rejected because: It retrieved semantically similar items but missed pricing-relevant nuances such as quantity differences that strongly affect price.

Direct title-to-price prediction only.

Rejected because: It improved MAE but could recommend visually and semantically unrelated cards, harming seller trust.

Patterns

rag-retrieval (retrieval)

eBay retrieves comparable historically sold items from a vector database to ground seller price guidance in actual sales.

retrieval-ranking (retrieval)

The system tunes embedding objectives so retrieved comparisons rank well for both semantic match and price relevance.

fine-tuning-adaptation (model_adaptation)

The team trains BERT-based domain-specific embeddings on eBay behavior and listing data instead of using generic encoders.

data-quality-loop (training_data)

User searches, clicks, price checks, structured-data checks, in-batch negatives, and hard negatives are used to convert marketplace behavior into training signal.

eval-driven-development (evaluation)

The team compares semantic similarity and title-to-price objectives on thousands of samples using MAE and semantic error behavior.

Tradeoffs

Use dense embedding retrieval for seller price guidance.

Chose Custom vector retrieval over historical sales over Keyword search over listing titles.

Rationale: Dense embeddings can learn domain equivalences and retrieve comparable sold items even when sellers use abbreviations or slang.

Train eBay-domain embeddings.

Chose Custom BERT-based encoders trained on marketplace data over Off-the-shelf general embeddings.

Rationale: eBay's domain-specific language and available behavior data made custom training more effective.

Use click behavior as training-pair signal.

Chose Search-session click pairs with validation rules over Manually labeled item similarity pairs only.

Rationale: User behavior supplies scalable weak supervision, while price and structured-data checks filter the pairs for real similarity.

Add hard negative mining.

Chose Similar-looking but differently priced negatives over In-batch random negatives alone.

Rationale: Hard negatives teach the model the nuanced pricing attributes that random negatives do not expose.

Use multi-task learning to navigate trust versus price accuracy.

Chose Shared model with semantic and price objectives plus alpha tuning over A single semantic-only or price-only objective.

Rationale: Multi-task learning allows stakeholders to decide how much MAE improvement is worth versus semantic matching errors.

Gotchas (what bit them)

Semantic similarity alone can miss price-changing attributes.

Root cause: Two listings can look textually similar while differing in quantity, grade, rarity, or condition enough to change price substantially.

Resolution: Introduce hard negatives and multi-task price prediction so the embedding space learns pricing-relevant distinctions.

Price prediction alone can create untrusted recommendations.

Root cause: Embeddings optimized for price can cluster unrelated items that sold for similar amounts.

Resolution: Keep semantic-matching quality in the objective and tune the alpha trade-off rather than optimizing MAE alone.

General embeddings underperform on domain-specific marketplace language.

Root cause: Sports-card titles use abbreviations, slang, and domain-specific terminology not captured well by general text encoders.

Resolution: Train custom BERT-based embeddings on eBay domain data when sufficient data and compute are available.

Random negatives are too easy for fine-grained pricing distinctions.

Root cause: Random in-batch negatives are usually unrelated, so they do not force the model to learn subtle attributes that separate similar cards.

Resolution: Mine hard negatives that appear similar but differ in price-relevant attributes such as grade or condition.

Evaluation

The team evaluates semantic similarity and title-to-price approaches on thousands of test samples, using MAE for pricing accuracy and semantic/player matching errors for seller trust. It then studies how alpha shifts the multi-task model along the trade-off curve.

Metrics: Mean Absolute Error for title-to-price prediction · Mean Absolute Error for semantic similarity retrieval · Correct-player or semantic matching errors · Trade-off curve as alpha moves between objectives · KNN retrieval quality over comparable sold items

Outcomes

• eBay built a retrieval-based price-guidance system for sellers based on comparable historical sales.
• Domain-trained BERT embeddings outperformed general off-the-shelf embeddings for marketplace terminology.
• Hard negative mining improved sensitivity to subtle pricing features.
• Title-to-price embeddings improved MAE to 29 versus 38 for semantic similarity, but introduced trust problems.
• Multi-task learning gave the business a tunable way to balance pricing accuracy and semantic trust.

The summary reports eBay scale at about 2 billion listings, more than 130 million active buyers, and 190 marketplace sites. The retrieval path must encode millions of historical listings and support efficient KNN search; the training/evaluation path uses thousands of test samples, batches of 64 with 63 in-batch negatives, and MAE comparisons of 29 versus 38.