Semantic Search for AI Agents at Scale: Retrieval and Ranking for LinkedIn's Hiring Assistant

linkedin · HR · 2026 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: AI recruiter sourcing and billion-scale semantic search

LinkedIn needed Hiring Assistant to translate natural-language recruiter requirements into qualified candidate matches across more than 1.3 billion member profiles, without the quality-coverage failures of keyword and faceted search.

Constraints:

• Recruiter queries are often paragraph-length natural-language descriptions rather than structured filters.
• Faceted search has high coverage but can surface unqualified profiles.
• Boolean keyword search can be more precise but returns zero results for nearly half of queries.
• The system must maintain sub-second serving, daily profile freshness, responsible AI constraints, and billion-profile retrieval scale.

Architecture

LinkedIn built MUSE-centered semantic retrieval for Hiring Assistant: a prompt-engineered LLM parses recruiter queries into structured role details and qualifications, MUSE dual-tower Matryoshka embeddings retrieve candidates from a 1B+ ANN index with ABM post-filters, results blend with other sourcing strategies in an L2 ranker, an LLM guard evaluates candidate fit, and a Lambda-style production pipeline maintains profile embeddings through full weekly rebuilds plus daily CDC-driven delta inference.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["Recruiter query"] --> B["Query understanding LLM"]
  B --> C["MUSE query tower"]
  D["Batch layer"] --> E["MUSE profile tower"]
  F["Speed layer"] --> E
  E --> G["Iceberg tables"]
  G --> H["ANN retrieval index"]
  G --> I["Venice feature store"]
  C --> H
  H --> J["ABM post-filters"]
  J --> K["L2 ranker"]
  C --> K
  I --> K
  K --> L["LLM candidate evaluator"]
  L --> M["Recruiter-facing results"]

Components

Data flow

A recruiter writes a natural-language query. Query understanding extracts structured details and qualifications. MUSE encodes the query, ANN retrieval scans precomputed profile embeddings, ABM post-filters narrow candidates, the L2 ranker blends and reorders sourcing strategies using full-dimensional MUSE features, and an LLM candidate evaluator screens and explains candidate-role fit before recruiters see results.

Why this architecture

LinkedIn separates query understanding, embedding retrieval, ranking, evaluation, and embedding-refresh infrastructure because no existing search strategy could satisfy both quality and liquidity at billion-profile scale, and Matryoshka embeddings let one model serve retrieval and ranking with different dimensional budgets.

Models used: MUSE Teacher · Expert Judge · MUSE dual-tower embeddings · open-weight reasoning model · proprietary LLM · vLLM · DCNv2 ranker

Scale

• Hiring Assistant searches over more than 1.3 billion member profiles.
  “look across more than 1.3 billion member profiles and find the candidates who actually match the query”article, stated
• Boolean search returned zero results for nearly half of queries before this work.
  “boolean (keyword) search offered better precision but returned zero results for nearly half of queries.”article, stated
• MUSE uses 2048-dimensional truncations for retrieval and full 4096-dimensional embeddings for ranking.
  “Retrieval needs to scan over 1B+ profiles via an ANN index. We use 2048-dimensional truncations here”article, stated

  “Ranking scores only the top N candidates returned by retrieval. We use the full 4096 dimensions as input features to the L2 ranker”article, stated
• The query embedding is served under 100ms at p95.
  “the query embedding is computed once via a hosted vLLM endpoint (under 100ms at p95)”article, stated
• A two-week A/B test showed +2.7% post-LLM Guard HRR and +4.1% InMail sends per recruiter seat.
  “A two-week A/B test validates this in production (here relative gains): post-LLM Guard HRR +2.7%, InMail Sends per seat +4.1%”article, stated
• MUSE retrieval reached 96% query coverage and about 76% retrieval pool utilization.
  “Its liquidity — ~76% average utilization of the retrieval pool at 96% query coverage — is on par with the prior high-liquidity faceted strategies”article, stated

Alternatives considered (roads not taken)

Faceted search

Rejected because: Faceted search answered many queries but returned many unqualified profiles.

Boolean keyword search

Rejected because: Boolean search had better precision but inadequate coverage for natural-language recruiter requirements.

Use generic embedding similarity without product-policy supervision

Rejected because: Generic similarity clusters semantically related profiles but does not encode the product definition of qualification fit.

Rely on exact kNN with post-filters

Rejected because: LinkedIn found ANN recall and ABM post-filter losses compound, motivating future pre-filtered infrastructure investment.

Patterns

retrieval-ranking (retrieval)

MUSE powers L1 semantic retrieval, then the L2 ranker blends, scores, and reorders candidates with MUSE embeddings as relevance features.

llm-as-judge (evaluation)

The Expert Judge and MUSE Teacher provide scalable qualification-fit labels and offline evaluation for semantic search.

model-distillation (model-adaptation)

LinkedIn compresses the Expert Judge's multi-step relevance reasoning into MUSE teacher labels and then into fixed-size dual-tower embeddings.

batch-offline-processing (infrastructure)

The batch layer performs full-corpus profile inference and weekly rebuilds, while daily CDC handles deltas.

safety-policy-enforcement (governance)

Qualification matching is governed by product policy and responsible AI principles encoded in the judge constitution.

Tradeoffs

Use Matryoshka embeddings at two dimensions.

Chose 2048-dimensional retrieval and 4096-dimensional ranking features over one embedding dimensionality for every stage.

Rationale: Retrieval needs speed over 1B+ profiles, while ranking can afford higher fidelity over fewer candidates.

Train on qualification-fit labels instead of engagement labels.

Chose MUSE Teacher labels governed by product policy over engagement-optimized retrieval supervision.

Rationale: Engagement signals do not directly encode qualified candidate-role fit.

Keep fewer but better candidate results.

Chose higher-quality candidate pool over maximizing candidate volume.

Rationale: Online results improved relevance and InMail sends while candidates sourced declined.

Gotchas (what bit them)

Search strategy tradeoffs broke under natural-language recruiter queries.

Root cause: Structured fields and keyword overlap do not capture paragraph-length qualifications or soft skills.

Resolution: Use LLM query understanding plus semantic embeddings trained on qualification-fit judgments.

Human labels are not always the ceiling for knowledge-intensive relevance judgments.

Root cause: Human annotators made mistakes on technical qualifications where high-confidence models had read the full context.

Resolution: Analyze disagreements and treat human labels as a noisy signal rather than an unquestioned ceiling.

ANN retrieval and post-filtering can multiply recall losses.

Root cause: Retrieved candidates may be filtered out after ANN search, wasting slots that exact pre-filtered search might have used for stronger matches.

Resolution: Introduce inline filtering and plan future infrastructure for pre-filtered exact or improved retrieval.

Billion-member embedding freshness is an infrastructure problem, not only a modeling problem.

Root cause: Recruiter results must reflect updated member data, new model versions, and regulatory deletions.

Resolution: Use Lambda architecture with weekly full rebuilds, daily CDC delta inference, and Iceberg tables as the source of truth.

Evaluation

LinkedIn evaluates MUSE through a cascade: PyTorch retrieval metrics on held-out data, Expert Judge replay of production queries comparing candidate models to control, and controlled online A/B tests measuring relevance, engagement, and sourcing throughput.

Metrics: Recall@k · MAP@k · NDCG@k · Highly Relevant Rate · Facepalm Rate · liquidity · candidate share · Eval Pass Rate · InMail Sends per seat · Viewed Contact Rate

Outcomes

• MUSE became the highest-quality sourcing strategy in Hiring Assistant.
• MUSE reached about 76% retrieval pool utilization at 96% query coverage.
• The full stack improved pre-LLM Guard HRR by +4% and reduced Facepalm rate by 5%.
• A two-week A/B test improved post-LLM Guard HRR by +2.7% and InMail sends per seat by +4.1%.
• Semantic retrieval candidate share rose from 18% to 31% after the MUSE-featured L2 ranker.

The system operates over 1B+ precomputed profile embeddings, daily CDC updates, weekly full rebuilds, and an online query embedding endpoint under 100ms p95.