LLMOps Lessons from W&B's Wandbot: Manual Evaluation & Quality Assurance of Production LLM Systems

weights_&_biases · Tech · 2023 · source Agentic: no · promotional bias: med · summary-only (no fetched source) · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Manual evaluation for production RAG assistants

Weights & Biases needed to assess the quality of Wandbot, an LLM-powered documentation assistant, in a setting where probabilistic LLM outputs and RAG failure modes make standard deterministic testing insufficient.

Constraints:

• The source content is limited and the summary explicitly says specific methodology details and results are not available.
• LLM outputs are probabilistic and can vary in subtle ways that are difficult to assess automatically.
• A RAG documentation assistant must evaluate both retrieval quality and generation quality.
• Manual evaluation is valuable for ground truth and edge-case discovery but is time-consuming, subjective, and hard to scale.
• Documentation-assistant quality depends on accuracy, completeness, relevance, groundedness, and clarity.

Architecture

The case describes Wandbot as a RAG-based documentation assistant and focuses on a manual evaluation layer around it. Human evaluators assess response quality dimensions such as accuracy, completeness, relevance, groundedness, and clarity, with manual review serving as ground truth, edge-case discovery, quality benchmarking, and stakeholder alignment. The summary does not provide enough source-backed detail to reconstruct a production topology, so the v2 architecture is intentionally sparse.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  wandbot-documentation-assistant["Wandbot documentation assistant"]
  retrieval-component["Retrieval component"]
  generation-component["Generation component"]
  manual-evaluation-workflow["Manual evaluation workflow"]
  wandbot-documentation-assistant --> retrieval-component
  wandbot-documentation-assistant --> generation-component
  wandbot-documentation-assistant --> manual-evaluation-workflow
  retrieval-component --> wandbot-documentation-assistant
  retrieval-component --> generation-component
  retrieval-component --> manual-evaluation-workflow
  generation-component --> retrieval-component
  generation-component --> wandbot-documentation-assistant
  generation-component --> manual-evaluation-workflow
  manual-evaluation-workflow --> wandbot-documentation-assistant
  manual-evaluation-workflow --> retrieval-component
  manual-evaluation-workflow --> generation-component

Components

Data flow

A user asks Wandbot documentation questions; the RAG system retrieves source documentation and generates an answer; human evaluators manually inspect outputs for correctness, completeness, relevance, groundedness, clarity, and whether failures come from retrieval or generation. The resulting judgments can be used as ground truth, benchmarks, and feedback for future automated evaluation.

Why this architecture

Manual evaluation is used because automated metrics alone cannot reliably judge all dimensions of documentation-answer quality, and RAG failures can originate in retrieval or generation. The design therefore treats human review as both a quality benchmark and a way to identify edge cases for later automation.

Models used: LLMs · RAG documentation assistant

Alternatives considered (roads not taken)

Rely only on automated metrics

Rejected because: The summary states manual evaluation is needed because automated systems can miss real-world LLM quality issues and edge cases.

Evaluate only end-to-end answer quality

Rejected because: RAG systems require separating retrieval failures from generation failures.

Patterns

eval-driven-development (evaluation)

The work centers evaluation as a core LLMOps practice for Wandbot, using manual assessment to establish ground truth, discover edge cases, benchmark quality, and support iterative improvement.

human-in-the-loop-gate (evaluation)

Human evaluators inspect Wandbot outputs for quality dimensions that are hard for automated metrics to assess reliably.

rag-retrieval (retrieval)

Wandbot is described as a RAG-based documentation assistant whose evaluation must separately assess retrieval and generation quality.

Tradeoffs

Use manual evaluation despite scalability limits

Chose Human judgment for quality benchmarking and edge-case discovery over Fully automated evaluation only.

Rationale: Manual review provides ground truth and finds real-world quality issues automated systems can miss, but should be paired with automation because it is expensive.

Evaluate RAG component quality separately

Chose Retrieval quality, generation quality, and end-to-end performance checks over A single aggregate chatbot-quality score.

Rationale: Component-level checks help identify whether failures come from missing documents or synthesis errors.

Gotchas (what bit them)

Automated metrics alone miss qualitative documentation-answer failures.

Root cause: LLM responses are probabilistic and subtle quality failures may not be captured by deterministic checks.

Resolution: Use manual evaluation to create ground truth, calibrate automated metrics, and discover edge cases.

RAG evaluation can hide whether retrieval or generation caused a bad answer.

Root cause: A RAG assistant has two coupled subsystems: document retrieval and answer synthesis.

Resolution: Assess retrieval independently, evaluate generation with good retrieval, and measure end-to-end performance.

Manual evaluation does not scale cleanly.

Root cause: Human review is expensive, subjective, inconsistent, and usually covers only sampled interactions.

Resolution: Use manual evaluation as a benchmark and calibration layer alongside automated evaluation.

Evaluation

The described evaluation approach is manual review of a RAG documentation assistant, assessing accuracy, completeness, relevance, groundedness, clarity, retrieval quality, generation quality, and end-to-end performance. The summary emphasizes that exact metrics and results are unavailable.

Metrics: Accuracy · Completeness · Relevance · Groundedness · Clarity · Retrieval quality · Generation quality · End-to-end performance

Outcomes

• W&B developed Wandbot and documented a manual evaluation approach for its LLM-powered documentation assistant.
• The available text supports the existence of a structured evaluation methodology but does not provide concrete performance results.
• The case highlights manual evaluation as a calibration and edge-case-discovery layer for RAG assistants.

No concrete operating scale, dataset size, or performance result is available from the source summary. The extraction is intentionally sparse and summary-only.