Optimizing RAG-based Search Results for Production: A Journey from POC to Production

statista · Research & Academia · 2023 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Research-data RAG search

Statista needed to turn a costly, low-quality Research AI proof of concept into a production RAG product for discovering statistics and insights.

Constraints:

• Statista serves 30,000 paying customers.
• The corpus contains millions of curated statistics.
• The POC made 42 LLM calls per request.
• Initial quality scores were only 30%.

Architecture

Statista added traceability, LLM-as-a-judge quality metrics, more than 100 experiments, multi-query rewriting, and HyDE to optimize Research AI for production.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  research-ai-interface["Research AI Interface"]
  query-rewriting-and-hyde["Query Rewriting and HyDE"]
  rag-pipeline["RAG Pipeline"]
  traceability-and-evaluation["Traceability and Evaluation"]
  research-ai-interface --> query-rewriting-and-hyde
  query-rewriting-and-hyde --> rag-pipeline
  rag-pipeline --> traceability-and-evaluation

Components

Data flow

User research questions are rewritten into multiple queries, HyDE generates hypothetical retrieval targets, the RAG pipeline retrieves statistics and insights, traces capture behavior, and LLM-as-a-judge metrics guide experiments.

Why this architecture

The production effort attacked both retrieval quality and operational cost through traceable experiments rather than accepting the expensive POC.

Models used: RAG · LLM-as-a-judge · HyDE · Multi-query rewriting

Scale

• Statista serves 30,000 paying customers.
  “serving 30,000 paying customers”summary, stated
• The POC made 42 LLM calls per request.
  “42 LLM calls per request”summary, stated
• Optimization took three weeks and more than 100 experiments.
  “conducted over 100 experiments”summary, stated
• The effort achieved 65% cost reduction, 10% latency improvement, and 140% quality improvement.
  “65% cost reduction, 10% latency improvement (with 65% savings reinvested in quality-boosting techniques), and 140% quality improvement”summary, stated

Alternatives considered (roads not taken)

Ship the proof of concept unchanged

Rejected because: It made 42 LLM calls per request and had only 30% quality.

Patterns

rag-retrieval (retrieval)

Research AI uses RAG over Statista's curated statistics.

query-rewriting (retrieval)

Multi-query rewriting and HyDE improve semantic retrieval.

llm-as-judge (evaluation)

LLM-as-a-judge metrics guided RAG optimization.

Tradeoffs

Reinvest savings into quality

Chose Use cost savings for quality-boosting techniques over Only minimizing cost.

Rationale: The system needed production quality as well as lower cost.

Gotchas (what bit them)

POC RAG quality and cost can be far from production-ready.

Root cause: The initial system made too many LLM calls and scored poorly.

Resolution: Add traceability, metrics, and many targeted retrieval experiments.

Evaluation

Statista used traceability, LLM-as-a-judge quality metrics, experiment counts, cost, latency, and quality improvement.

Metrics: Quality score · Cost reduction · Latency improvement · Experiment count

Outcomes

• Cost fell 65%.
• Latency improved 10%.
• Quality improved 140%.
• The system became production-ready in three weeks.

Statista serves 30,000 paying customers and millions of curated statistics.