Scaling Meta AI's Feed Deep Dive from Launch to Product-Market Fit

meta · Media & Entertainment · 2024 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: social feed AI feature scaling

Meta needed to take Feed Deep Dive from launch to product-market fit while giving Facebook users high-quality contextual answers at Meta scale.

Constraints:

LLMs can hallucinate or refuse, harming engagement and trust.
Feed Deep Dive competes with simply continuing to scroll the Facebook feed.
Quality improvements introduce latency and capacity costs.

Architecture

Meta evolved Feed Deep Dive toward agentic models with planning, tool calling, and reflection; used auto-judges over production traffic to detect refusals; pushed refusal feedback into the prompt-ranking model; applied smart caching on high-traffic posts and topic/intent-aware TTLs; and used cohort/content/funnel analysis to target users who get the most value.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  A["Facebook feed post"] --> B["Agentic planning model"]
  B --> C["Tool-calling layer"]
  C --> D["Reflection step"]
  D --> E["Response generator"]
  E --> F["Smart cache"]
  G["Auto-judge pipeline"] --> H["Prompt ranking model"]
  H --> A
  I["Cohort targeting model"] --> A

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Facebook feed post`	User-facing content that may trigger information-seeking questions.	`Agentic planning model`
`Agentic planning model`	Creates a plan for which tools/APIs to call.	`Facebook feed post`, `Tool-calling layer`
`Tool-calling layer`	Calls people/entity, multimodal, and similar-post APIs for context.	`Agentic planning model`, `Reflection step`
`Reflection step`	Checks whether gathered context and proposed response are good enough.	`Tool-calling layer`, `Response generator`
`Response generator`	Produces the contextual Feed Deep Dive answer.	`Reflection step`, `Smart cache`
`Auto-judge pipeline`	Continuously evaluates production traffic for refusals and quality issues.	`Prompt ranking model`
`Prompt ranking model`	Ranks prompts/questions to show, downranking prompts likely to cause refusals.	`Auto-judge pipeline`, `Facebook feed post`
`Smart cache`	Caches responses for high-traffic posts with topic/intent-aware TTLs.	`Response generator`
`Cohort targeting model`	Increases or reduces feature prevalence for cohorts based on value and engagement.	`Facebook feed post`

Data flow

A feed post triggers a contextual question. The agentic flow plans which APIs/tools to call, gathers people/multimodal/similar-post context, reflects on whether the answer is good, and returns a response. Auto-judges continuously evaluate production prompts, rankers downrank prompts likely to cause refusals, caches store responses for high-traffic posts with TTL by topic/intent, and cohort models decide where to show the feature.

3 supporting quotes

“An agentic model really has three key features. It leverages planning. Then it does tool calling and then it does reflection.”transcript, stated

“We capture the refusals user as feedback and push the feedback not to the prompt generation model but to the prompt ranking model”transcript, stated

“we have about 5 to 10% of posts actually driving the vast majority about 50% of the the impressions and 30% of clicks.”transcript, stated

Why this architecture

Meta optimized Feed Deep Dive as a product loop: improve response quality with agentic reasoning, reduce refusals with judges/rankers, offset latency/cost with caching, and target cohorts where the feature creates value.

3 supporting quotes

“leveraging techniques like agentic models reasoning and then scaling this up by improving our latency reducing our capacity costs”transcript, stated

“refusals are not great and we need to find ways to reduce them.”transcript, stated

“for users who are not getting value we roll back or or squeeze back the feature”transcript, stated

Models used: Meta AI · agentic planning model · auto judges · prompt ranking model · deep neural networks for targeting

Scale

Feed Deep Dive launched in April 2024.
“in April 2024 we launched Meta AI on Facebook”transcript, stated
After US English launch, Meta expanded to Spanish regions, languages, and regions around the world.
“we launched in the US in English. Then a couple of months after that we launched in Spanish speaking regions”transcript, stated
Roughly 5-10% of posts drive about 50% of impressions and 30% of clicks.
“5 to 10% of posts actually driving the vast majority about 50% of the the impressions and 30% of clicks.”transcript, stated

Alternatives considered (roads not taken)

Traditional orchestration without agentic planning/reflection

Rejected because: Meta moved to planning, tool calling, and reflection for higher-quality contextual responses.

“this is the problem with using traditional orchestration. What we can do is we can start to move to agentic approaches.”transcript, stated

Offline model improvement only

Rejected because: Model improvement can take weeks, while bad prompts may be prevalent only for days.

“this model improvement cycle is often going to take a few weeks to actually get to fine-tune the model”transcript, stated

Uniform cache TTL for all topics

Rejected because: Fast-moving sports/news need freshness while how-tos and recipes tolerate longer TTLs.

“sports and news require a very short TTL since the answer response will keep is in a fastmoving environment.”transcript, stated

Patterns

`tool-use-orchestration` (agent-architecture)

Feed Deep Dive plans and calls APIs/tools to gather context before answering.

“we'll first generate a a plan of what tools to call and tools are just APIs.”transcript, stated

`llm-as-judge` (evaluation)

Auto judges evaluate production traffic for refusal behavior and feed prompt-ranker improvement.

“We write a few auto judges that evaluate whether the response was a refusal.”transcript, stated

`semantic-caching` (serving)

Smart caching targets high-traffic posts and uses topic/intent TTLs to balance freshness and latency.

“how do we scale this up and how do we scale up judges and how do we scale up context enrichment. The one single answer here apart from many many tricks in the book are is actually smart caching.”transcript, stated

`retrieval-ranking` (ranking)

Feedback from refusals is pushed to the prompt ranking model so poor prompts are downranked.

“push the feedback not to the prompt generation model but to the prompt ranking model”transcript, stated

Tradeoffs

Reasoning architecture

Chose agentic planning, tool calling, and reflection over traditional orchestration.

Rationale: The agentic approach provides higher-quality context-enriched responses.

“come up with a proposed response, reflect on it and based on the reflection, if it looks good, then we show it to the user.”transcript, stated

Cache policy

Chose topic/intent-aware TTLs over one freshness setting for all posts.

Rationale: Freshness needs differ between news/sports and more evergreen how-to/recipe topics.

“try to figure out what topics need a short latency or a short TTL and what topics require are are able to tolerate a long TTL.”transcript, stated

Feature distribution

Chose cohort targeting and rollback over showing Feed Deep Dive equally to every user.

Rationale: Meta increases prevalence for users getting value and reduces it where the feature does not work.

“giving these users who are getting value similar such users to have more and more prevalence of the feature”transcript, stated

Gotchas (what bit them)

Refusals break habit and trust.

Root cause: LLMs refuse when context is missing, context generation fails, or integrity/safety requires it.

Resolution: Use auto judges and prompt-ranker feedback to reduce refusal-prone prompts.

“each time we we show a refusal it leads to disengagement and it breaks habits for the user and breaks effective trust”transcript, stated

Agentic reasoning improves quality but adds latency.

Root cause: Planning, tool calls, and reflection require extra steps before showing a response.

Resolution: Use caching and product targeting to manage quality/latency tradeoffs.

“Of course, this is now bringing in latency, but there's a higher quality response”transcript, stated

Evaluation

Meta evaluates through auto judges, human raters, refusal rates, prompt-ranking feedback, click-through rates, deep sessions, weekly active usage, L7/L28 retention, cohort performance, latency, capacity cost, and freshness.

Metrics: refusal rate · auto-judge/human disagreement · click-through rate · deep sessions · weekly active usage · L7/L28ness · cache hit rate · latency · capacity cost

“We sample some of them using human raiders and then we do a comparative analysis”transcript, stated

“click-through rates, deep sessions, weekly active usage, L7ness or L28ness”transcript, stated

Outcomes

Feed Deep Dive launched in April 2024 and expanded beyond US English.
Meta built an online auto-judge and prompt-ranker feedback pipeline for refusals.
Smart caching focuses on the small share of posts driving large impression/click volume.
The product moved toward product-market fit and monetization work.

The transcript frames outcomes as product-market-fit progress rather than disclosing absolute engagement or revenue numbers.

“how do we build out a launch product and take it all the way to product market fit”transcript, stated

“moving on to monetization. So for”transcript, stated