Building a Complex AI Answer Engine with Multi-Step Reasoning

perplexity · Tech · 2024 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Multi-step AI answer search

Perplexity Pro Search needed to answer complex questions that traditional search handles poorly because they require decomposing a nuanced query, researching multiple subquestions, filtering documents, and synthesizing a factual response.

Constraints:

complex queries require connecting multiple ideas
speed and accuracy both matter despite added reasoning steps
different language models interpret prompts differently
users need visibility into intermediate progress when waiting
evaluation must cover answer quality, intermediate steps, and UX

Architecture

Perplexity separates planning from execution. The agent creates a step-by-step plan, generates search queries for each step, executes those steps sequentially while passing prior results forward, groups and filters returned documents, and sends highly ranked evidence to an LLM for final answer synthesis. The product also exposes progress and citations in the UI.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  user-query[User query] --> planning-agent[Planning agent]
  planning-agent --> search-executor[Search executor]
  search-executor --> document-filter[Document filter]
  document-filter --> answer-synthesizer[Answer synthesizer]
  answer-synthesizer --> progress-ui[Progress UI]
  progress-ui --> evaluation-loop[Evaluation loop]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`User query`	Complex question submitted to Pro Search.	`Planning agent`
`Planning agent`	Creates a step-by-step plan for answering the query.	`User query`, `Search executor`
`Search executor`	Generates and runs search queries for each plan step.	`Planning agent`, `Document filter`
`Document filter`	Groups and filters returned documents to the most relevant set.	`Search executor`, `Answer synthesizer`
`Answer synthesizer`	LLM step that writes the final answer from ranked documents.	`Document filter`, `Progress UI`
`Progress UI`	Shows plan execution, expandable steps, and citation snippets.	`Answer synthesizer`, `Evaluation loop`
`Evaluation loop`	Manual tests, dogfooding, LLM-as-judge rankings, and A/B tests.	`Progress UI`

Data flow

A complex user query enters Pro Search. The agent builds a plan, emits search queries for each step, runs them sequentially, groups and filters documents, passes the best-ranked documents to an LLM, and returns an answer with visible intermediate progress and citations.

3 supporting quotes

“When a user submits a query, the AI creates a plan— a step-by-step guide to answering it. For each step in the plan, a list of search queries are generated and executed.”article, stated

“For each step in the plan, a list of search queries are generated and executed. These steps are executed sequentially, and results from previous steps are passed when executing steps after.”article, stated

“The highly-ranked documents are then passed to an LLM to generate a final answer. Perplexity Pro Search also supports specialized tools such as code interpreters, which allow users to run calculations or analyze files on the fly, as well as mathematics evaluations tools like Wolfram Alpha.”article, stated

Why this architecture

Planning separates reasoning about the research path from executing searches, which improves multi-step coverage. Showing intermediate progress makes users more willing to wait for a slower but more capable search mode.

“Perplexity Pro’s AI agent separates planning from execution, which yields better results for multi-step search.”article, stated

“Perplexity found that users were more willing to wait for results if the product would display the intermediate progress.”article, stated

Models used: large language models · LLM-as-judge · Wolfram Alpha · code interpreter

Scale

Pro Search query volume increased by over 50 percent in the past few months.
“###### In fact, query search volume of Perplexity Pro Search has increased by over 50% _in the past few months, as more users discover its ability to answer tricky questions quickly and efficiently.”article, stated

Alternatives considered (roads not taken)

Traditional search

Rejected because: Traditional search struggles with queries that require connecting details across multiple ideas.

“Traditional search engines may struggle to answer complex queries that require connecting the dots across multiple ideas”article, stated

One quick-search mode for all questions

Rejected because: Pro Search targets complex, nuanced questions that need a more advanced modality.

“Unlike Perplexity’s quick search, which is designed for off-the-cuff questions, this advanced modality helps students, researchers, and enterprises”article, stated

Patterns

`prompt-chaining` (reasoning)

Pro Search executes a plan through sequential search steps before final synthesis.

“These steps are executed sequentially, and results from previous steps are passed when executing steps after.”article, stated

`retrieval-ranking` (retrieval)

Search documents are grouped, filtered, and passed to the LLM only after ranking.

“documents, which are grouped and then filtered down to the most relevant ones.”article, stated

`llm-as-judge` (evaluation)

Perplexity uses LLM-as-a-judge rankings to scale answer evaluation.

“used an LLM-as-a-Judge to rank the answers.”article, stated

Tradeoffs

Search mode

Chose multi-step planning and execution over off-the-cuff quick search.

Rationale: Complex questions need decomposed research to produce precise answers.

“This is where Perplexity Pro Search shines. Their AI agent breaks down multi-step questions”article, stated

Product configuration testing

Chose A/B tests over latency and cost configurations over single hand-picked configuration.

Rationale: Different model choices trade off latency, cost, and user reaction.

“A/B tests were run on users to gauge their reactions to different possible configurations of the product, such as tradeoffs between latency and costs”article, stated

Gotchas (what bit them)

Long complex prompts can become hard for models to follow.

Root cause: Prompt rules and examples must be balanced against prompt length.

Resolution: Iterate on few-shot examples, chain-of-thought prompting, and intermediate-step checks.

“It’s harder for models to follow the instructions of really complex prompts.”article, stated

Users may abandon slower multi-step search if progress is opaque.

Root cause: Advanced search takes longer than quick search.

Resolution: Show the plan executing step by step with expandable details and citation hover states.

“This led to the development of an interactive UI that shows the plan being executed step-by-step.”article, stated

Evaluation

Perplexity combines manual side-by-side testing, internal dogfooding, intermediate-step inspection, LLM-as-judge ranking, and A/B tests for latency/cost configurations.

Metrics: answer quality · UX quality · LLM-as-judge rank · A/B test reaction · query search volume

“The team conducted manual evaluations by testing Pro Search on a wide range of queries and comparing its answers side-by-side”article, stated

Outcomes

Pro Search query volume increased by over 50 percent.
The product delivered multi-step answers while preserving speed and accuracy.
Intermediate progress and citations improved the user experience for longer tasks.

The source reports query volume growth but not absolute traffic volume.

“they didn’t have to compromise on speed and accuracy, even when increasing the complexity of its systems.”article, stated