Advanced Embedding-Based Retrieval for Personalized Content Discovery

pinterest · Tech · 2024 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.2 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Personalized content retrieval

Pinterest needed stronger Homefeed retrieval to deliver personalized, engaging, and diverse candidate content.

Constraints:

• Two-tower retrieval simplicity
• Serving corpus quality limits
• Diverse user intents

Architecture

Pinterest's learned-retrieval system is a production candidate generator for Homefeed. A two-tower model computes user and Pin embeddings, with the Pin tower run offline and the user tower fetched once per request; MaskNet/DHEN-style feature crossing improves the user tower; fixed ID embeddings and dropout stabilize scaling; serving-corpus changes remove stale or remapped content; multi-embedding clustering represents diverse user interests; ANN search retrieves candidates that are combined round-robin and handed to ranking and blending. Conditional retrieval adds an interest input for product surfaces that need more control.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  homefeed-request[Homefeed request] --> user-tower[User tower]
  feature-crossing-modules[Feature crossing modules] --> user-tower
  feature-crossing-modules --> pin-tower[Pin tower]
  id-embedding-table[ID embedding table] --> user-tower
  pin-tower --> serving-corpus[Serving corpus]
  user-tower --> multi-embedding-interest-clusters[Multi-embedding interest clusters]
  conditional-retrieval-input[Conditional retrieval input] --> user-tower
  serving-corpus --> ann-search[ANN search]
  multi-embedding-interest-clusters --> ann-search
  ann-search --> round-robin-combiner[Round-robin combiner]
  round-robin-combiner --> ranking-and-blending-layers[Ranking and blending layers]
  ranking-and-blending-layers --> homefeed-request

Components

Data flow

Offline Pin-tower computation creates Pin embeddings and a curated serving corpus. At request time the user tower builds one or more user-interest embeddings from history, ANN search retrieves Pins for each embedding, results are combined round-robin, and ranking/blending layers produce Homefeed candidates. Conditional retrieval can add an interest token or filter to focus retrieval on a chosen interest.

Why this architecture

Pinterest chose learned retrieval because Homefeed needs candidate generation that is personalized, engaging, and diverse at large scale. The two-tower split allows heavier offline Pin computation and one online user-tower fetch; corpus renovation and multi-embedding retrieval address freshness, upper-bound quality, and diversity without relying only on one user vector.

Models used: two-tower model · MaskNet · DHEN

Scale

• The model-architecture upgrade improved engaged sessions by 0.15-0.35%.
  “This model architecture upgrade improved 0.15–0.35% engaged sessions across Pinterest. Press enter or click to view image in full size Figure 1.”article, stated
• Scaling the model added 0.1-0.2% engaged sessions and more than 1% Homefeed saves and clicks.
  “This scaling up brings another +0.1–0.2% engaged sessions, together with >1% homefeed saves and clicks. Press enter or click to view image in full size Figure 2.”article, stated
• Serving-corpus changes produced 0.1-0.2% engaged-session gains without model changes.
  “Closing this gap with a dedicated image signature remapping logic plus the time decay heuristics, we achieved +0.1–0.2% engaged sessions without any modeling changes.”article, stated

Alternatives considered (roads not taken)

Train a larger sparse ID embedding table directly

Rejected because: The larger table overfit badly, so Pinterest fixed the table and used aggressive dropout first.

Use one embedding for all user interests

Rejected because: Pinterest introduced multiple embeddings and clustering to represent diverse interests and combine retrieval results.

Improve retrieval only through modeling

Rejected because: The serving corpus itself set the ceiling, and corpus renovation improved metrics without model changes.

Patterns

rag-retrieval (retrieval)

The system is a learned retrieval candidate generator for personalized feed content.

retrieval-ranking (retrieval)

Pinterest improved retrieval by changing model architecture and serving-corpus construction.

Tradeoffs

How much model complexity to put in each tower.

Chose complicated user and Pin towers over simpler towers with less feature crossing.

Rationale: The offline Pin tower and once-per-request user tower made heavier structures operationally feasible.

How to improve candidate diversity.

Chose multiple clustered user embeddings with round-robin combination over a single user embedding feeding ANN search.

Rationale: Multiple embeddings let retrieval cover several user interests and then combine their candidates downstream.

Gotchas (what bit them)

ID embedding transfer risked overfitting.

Root cause: Direct fine-tuning did not perform well online.

Resolution: Pinterest mitigated overfitting with pretraining and contrastive-learning design.

Evaluation

Pinterest evaluated online engagement changes from retrieval architecture and corpus updates.

Metrics: engaged sessions · repins · clicks

Outcomes

• Feature crossing, ID embeddings, and retrieval changes produced measurable engagement gains.

Reported gains include engaged sessions, repins, and clicks.