Building Trust in RAG Systems Through Structured Feedback and User Collaboration

needl.ai · Finance · 2025 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: enterprise RAG trust and QA

AskNeedl's RAG answers could be technically plausible yet lose user trust because of missing citations, incomplete answers, wrong references, vague wording, and insufficient auditability for compliance and market-analysis users.

Constraints:

• Enterprise users need traceability to source documents, not just plausible answers.
• Many broad enterprise-search questions have no single gold answer.
• The team did not have an automated hallucination evaluator or benchmark suite at the start.
• Quality issues had to be prioritized across high-frequency queries, high-trust personas, and high-sensitivity topics.
• The improvement path explicitly avoided retraining the underlying model.

Architecture

Needl.ai created a structured QA loop around AskNeedl: log queries from internal and pilot sessions, annotate suspicious cases by failure type, build themed QA sets, use early adopters as ground truth, translate user feedback into ML-actionable retrieval/prompt/citation changes, partially automate evaluation with MCP, and sample live usage behavior for trust signals.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  askneedl-rag-system["AskNeedl RAG system"] --> query-logging["query logging"]
  askneedl-rag-system --> citation-surface["citation surface"]
  query-logging --> failure-annotation-taxonomy["failure annotation taxonomy"]
  failure-annotation-taxonomy --> shared-qa-sheet["shared QA sheet"]
  failure-annotation-taxonomy --> themed-qa-sets["themed QA sets"]
  early-adopter-users["early adopter users"] --> pm-translation-layer["PM translation layer"]
  early-adopter-users --> human-evaluation-layer["human evaluation layer"]
  shared-qa-sheet --> pm-translation-layer
  pm-translation-layer --> ml-iteration-loop["ML iteration loop"]
  themed-qa-sets --> ml-iteration-loop
  live-usage-review["live usage review"] --> human-evaluation-layer
  human-evaluation-layer --> ml-iteration-loop
  ml-iteration-loop --> askneedl-rag-system
  mcp-qa-automation["MCP QA automation"] --> reports-and-dashboards["reports and dashboards"]
  reports-and-dashboards --> decision-systems["decision systems"]

Components

Data flow

User and internal AskNeedl queries are logged; failed or suspicious cases are annotated as hallucination, citation missing, partial answer, or retrieval gap; a shared sheet links correct documents and themes; early users judge trust and sufficiency; the PM translates user-language complaints into retrieval, prompt, and citation work for ML; MCP-assisted QA and live usage reviews feed the next improvement loop.

Why this architecture

Needl.ai chose a structured product-and-user feedback loop because the failures were trust and auditability failures, not simply model errors. That made real-user judgment, citation coverage, retrieval tuning, and PM-to-ML translation more useful than immediate model retraining or fully automated scoring.

Models used: ChatGPT · Perplexity · Claude

Scale

• Needl.ai built approximately 200 task-specific evaluation queries.
  “We created a bank of ~200 task-specific queries”article, stated
• The feedback loop used a few early adopter teams from compliance, market analysis, and documentation roles.
  “compliance officers, market analysts, and documentation experts — who were already using AskNeedl in production or pilot settings.”article, stated
• The QA loop was initially manual but later partially automated with MCP.
  “to partially automate response evaluation — adding structure to what was previously a manual loop.”article, stated
• Needl.ai prioritized fixes by frequency, trust-critical personas, and sensitive financial/regulatory topics.
  “High-frequency queries with repeat issues”article, stated

  “High-trust personas (e.g., compliance teams, investor relations) who needed grounded outputs”article, stated

Alternatives considered (roads not taken)

Retrain the underlying model.

Rejected because: The post explicitly frames the work as progress without retraining, focusing instead on retrieval, prompts, citations, and feedback loops.

Depend on an automated hallucination evaluator or benchmark suite.

Rejected because: The team did not have those tools, so they built a manual high-signal QA loop first.

Use one-liner correctness scores.

Rejected because: Needl.ai found RAG quality required multiple dimensions such as completeness, factuality, relevance, trustworthiness, faithfulness, coverage, and confidence calibration.

Treat all failures as equally urgent.

Rejected because: The PM prioritized by repeat frequency, trust-critical personas, and high-sensitivity topics.

Patterns

rag-retrieval (retrieval)

AskNeedl answers enterprise search questions from retrieved documents and source citations.

eval-driven-development (evaluation)

The team built themed QA sets, task-specific queries, live usage reviews, and human evaluation loops to guide system improvements.

human-in-the-loop-gate (quality assurance)

Early users judged what counted as trustworthy, acceptable paraphrasing, adequate citations, and grounded answers.

observability-tracing (production monitoring)

The team logged queries, reviewed live user sessions, watched reformulation and citation-click behavior, and used those traces as quality signals.

retrieval-ranking (retrieval)

Feedback drove retrieval strategy changes such as prioritizing recent disclosures and relevant document types.

Tradeoffs

Improve RAG behavior without retraining.

Chose Retrieval, prompt, citation, fallback, and feedback-loop improvements over Model retraining.

Rationale: The team could target trust defects with structured examples and product/ML collaboration instead of changing model weights.

Use lightweight manual QA before full automation.

Chose Shared sheets, annotations, themed QA sets, and user review over Automated hallucination evaluator or gold benchmark suite.

Rationale: The team lacked automated tools but had user feedback and domain knowledge, so manual structure produced actionable signal quickly.

Prioritize trust-critical cases rather than all failures equally.

Chose High-frequency, high-trust-persona, and high-sensitivity fixes over Undifferentiated defect cleanup.

Rationale: Business-critical trust restoration mattered more than model elegance or exhaustive fixing.

Gotchas (what bit them)

Correct-looking RAG answers can still break trust.

Root cause: Missing citations, vague wording, incomplete coverage, or wrong references can make answers feel inferred rather than grounded.

Resolution: Evaluate trust, auditability, citation coverage, and completeness rather than factuality alone.

User complaints need translation before ML teams can act.

Root cause: User language such as 'this feels vague' does not directly identify retrieval, entity-matching, or citation defects.

Resolution: Use a PM translation layer to turn user trust signals into concrete ML and product work.

Many enterprise-search questions cannot be scored with a single correct answer.

Root cause: Queries about risks, allegations, or outlooks are broad and open-ended.

Resolution: Use multi-dimensional evaluation and human judgment instead of binary correctness.

Full automation came too early would miss contextual trust judgments.

Root cause: Trustworthiness depended on user roles, document context, and subjective-but-important signals.

Resolution: Start with disciplined human feedback and only partially automate response evaluation once structure exists.

Evaluation

Needl.ai used task-specific query banks, live usage review, early-user trust questions, and multi-dimensional tags for grounding, retrieval completeness, and confidence calibration.

Metrics: ~200 task-specific queries · Expected behavior and red flags per query · Reformulation behavior · Citation visibility and citation clicks · Human trust-in-report questions · Tags such as fully grounded with citation, partially retrieved, fluent but unverifiable, factually incorrect or hallucinated

Outcomes

• The team converted scattered user frustration into structured issues and themed QA sets.
• The ML team made targeted improvements to retrieval strategy, prompts, citation formatting, and fallback logic without retraining.
• MCP partially automated internal response evaluation and connected grounded answers to reports, dashboards, and decision systems.

The case discloses ~200 task-specific queries and early-adopter production/pilot usage, but not quantitative lift in trust, latency, or answer quality.