Implementing RAG and RagRails for Reliable Conversational AI in Insurance

geico · Insurance · 2023 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Reliable customer-facing conversational AI for insurance workflows

GEICO explored a conversational chat application that could collect user information instead of relying on web forms, but commercial and open-source LLMs varied without constraints, hallucinated plausible but false answers, and overpromised capabilities they did not have. In an insurance setting, the team needed a way to ground answers, guide the model through process state, limit false system-capability assumptions, and evaluate repeatability before using generative technology safely.

Constraints:

• the work began as an internal hackathon and experimental RAG implementation
• public-facing customer use cases require consistency, reliability, and compliance
• LLM responses vary without guardrails or constraints
• adding too much context can increase response variability and cost
• retrieved passages can be ignored when placed in the middle of the prompt
• permanent system-prompt instructions disrupted other objectives
• higher truthfulness and precision can require higher inference cost

Architecture

GEICO built an experimental RAG architecture for a conversational insurance chat application. Business data is split into records, embedded through an API, enriched with LLM-extracted metadata, and indexed in a vector database through an offline asynchronous conversion path. At interaction time, the system dynamically composes a system prompt from the conversation state, quote process stage, task description, constraints, and RAG context. User messages are also translated by an LLM into coherent likely questions, causing retrieval over both original and translated inputs. Retrieved records are ranked and reordered to improve model focus, filtered by an LLM relevance check, inserted into the prompt as answer-only context where possible, and augmented with RagRails instructions when relevant records need to steer the model away from overpromising.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  Customer_user["Customer user"] --> Conversational_chat_application["Conversational chat application"]
  Conversational_chat_application --> Conversation_state_tracker["Conversation state tracker"]
  Conversation_state_tracker --> Dynamic_system_prompt_composer["Dynamic system prompt composer"]
  Business_knowledge_sources["Business knowledge sources"] --> Document_splitting_pipeline["Document splitting pipeline"]
  Document_splitting_pipeline --> Embedding_API["Embedding API"]
  Document_splitting_pipeline --> Metadata_extraction_LLM["Metadata extraction LLM"]
  Embedding_API --> Offline_indexing_builder["Offline indexing builder"]
  Metadata_extraction_LLM --> Offline_indexing_builder
  Offline_indexing_builder --> HNSW_vector_database["HNSW vector database"]
  Conversational_chat_application --> Input_translation_LLM["Input translation LLM"]
  Input_translation_LLM --> RAG_retrieval_runtime["RAG retrieval runtime"]
  HNSW_vector_database --> RAG_retrieval_runtime
  RAG_retrieval_runtime --> Ranking_mechanism["Ranking mechanism"]
  RAG_retrieval_runtime --> Relevance_checker["Relevance checker"]
  RAG_retrieval_runtime --> RagRails_records["RagRails records"]
  Ranking_mechanism --> Dynamic_system_prompt_composer
  Relevance_checker --> Dynamic_system_prompt_composer
  RagRails_records --> Dynamic_system_prompt_composer
  Dynamic_system_prompt_composer --> LLM_response_generator["LLM response generator"]
  LLM_response_generator --> Conversational_chat_application
  LLM_response_generator --> Repeatability_test_harness["Repeatability test harness"]
  RagRails_records --> Repeatability_test_harness

Components

Data flow

Dense business knowledge is split, embedded, given metadata, and indexed offline into HNSW-backed vector collections. During a chat, GEICO composes state-aware instructions and retrieves records based on both the user's raw input and an LLM-translated version of the input. It combines the retrieved record sets, ranks and positions the most relevant knowledge, optionally checks relevance, and injects focused answer context into the system prompt. When a retrieved record carries a RagRails instruction, that instruction is applied only in relevant contexts to constrain model behavior and reduce overpromising.

Why this architecture

GEICO chose RAG before fine-tuning because it was cheaper, more flexible, more transparent, and easier to update. Offline indexing isolates expensive vector-structure construction from retrieval QPS. Query translation and ranking address messy user language and lost-in-the-middle behavior. RagRails solves a prompt-control problem that permanent system-prompt instructions could not solve cleanly by attaching the behavioral guidance to relevant retrieved records only.

"We considered RAG as a first line of defense due to:

• Cost: Avoids expensive model retraining."

Models used: GPT model · commercial LLMs · open-source LLMs · embedding model/API · LLM for metadata extraction · LLM for input translation · LLM for relevance checking

Scale

• The effort began in a 2023 GEICO tech hackathon.
  “In 2023, GEICO invited the tech organization to a hackathon designed to help us identify how and where LLMs could be used”article, stated
• The team measured the overpromising problem over 20 repeated responses.
  “In this case, 12 out of 20 responses were incorrect during repeated testing.”article, stated
• RagRails reduced overpromising errors from 12 to 6 and eventually to none in the test scenario.
  “This approach reduced the 12 errors to 6, and eventually to none after further adjustments, leading to our RagRails strategy.”article, stated
• The design explicitly optimized for Queries per Second by separating indexing from retrieval.
  “This approach aims to maximize Queries per Second (QPS) without the load of indexing during retrieval.”article, stated

Alternatives considered (roads not taken)

Fine-tune as the first hallucination mitigation step

Rejected because: GEICO treated fine-tuning as a last resort because RAG was cheaper, flexible, transparent, and efficient for updating knowledge.

• Transparency: Enables easy interpretability and attribution."

— article, stated

Use the entire retrieved record in the system prompt

Rejected because: Including full records was ineffective and unreliable; inserting only the answer portion produced more focused outcomes.

Retrieve only from the user's raw message

Rejected because: Everyday language was fragmented and varied, so GEICO also translated the input into a coherent likely question and retrieved both sets.

Permanently add all behavioral instructions to the system prompt

Rejected because: Those instructions disrupted other goals; RagRails attaches guidance to retrieved relevant records instead.

Use the largest LLM for every subtask

Rejected because: GEICO suggests smaller, more finely tuned models for optimization, extraction, relevance, and validation, with the LLM as backup.

Patterns

rag-retrieval (knowledge-architecture)

GEICO converts business knowledge into embedded records and retrieves authoritative context into the system prompt instead of relying on model memory.

batch-offline-processing (operations)

The vector indexing and snapshot-building work happens offline and asynchronously so retrieval can serve traffic without indexing load.

retrieval-ranking (retrieval)

GEICO ranks and reorders retrieved records so the most relevant material sits at the beginning or end of the context window.

query-rewriting (retrieval)

Every user message is rewritten into a coherent likely question for better knowledge-base matching.

llm-as-judge (quality)

GEICO uses the same LLM to evaluate retrieved record relevance before injecting too much context.

guardrails-io-filtering (quality)

RagRails places guidance inside retrieved records to steer the model away from incorrect capability claims.

cost-aware-routing (operations)

GEICO proposes smaller specialized models for subtasks and using the LLM as backup to reduce inference burden.

Tradeoffs

RAG before fine-tuning

Chose retrieval-augmented generation as first line of defense over fine-tuning as the initial mitigation.

Rationale: RAG avoided retraining cost, supported independently updated knowledge, improved transparency, and allowed rapid knowledge access.

"We considered RAG as a first line of defense due to:

• Cost: Avoids expensive model retraining."

— article, stated "* Transparency: Enables easy interpretability and attribution.

• Efficiency: Supports rapid knowledge access."

— article, stated

Offline indexing

Chose asynchronous collection building and snapshots over doing vector-index construction during retrieval.

Rationale: Vector indexing was resource intensive, so separating it from retrieval protected QPS and reduced disruption.

Answer-only prompt insertion

Chose insert the answer portion of records over insert entire records including examples.

Rationale: Focused insertion reduced ineffective and unreliable behavior from noisy record context.

Contextual RagRails

Chose instructions embedded in retrieved relevant records over permanent global instructions in the system prompt.

Rationale: Record-level guidance constrained behavior only when relevant without disrupting other objectives.

Cost versus truthfulness

Chose higher inference context for truthfulness-critical scenarios over minimizing inference cost at the expense of consistency.

Rationale: GEICO argues dependable application should be prioritized in scenarios requiring high truthfulness and precision.

Gotchas (what bit them)

LLMs overpromise capabilities in customer-facing flows.

Root cause: A helpful persona plus RAG context can make the model assume it can perform actions related to an answer.

Resolution: Attach guiding instructions to relevant retrieved records using RagRails and test repeatability.

Full records in prompts were noisy and unreliable.

Root cause: Examples and record structure inside the prompt confused the model.

Resolution: Delineate record structure and insert only the answer portion.

More retrieved context increased variability and cost.

Root cause: Adding extra context can produce response variability while increasing inference burden.

Resolution: Introduce an LLM relevance check and consider whether contexts are actually applicable before insertion.

Retrieved passages can be missed in the middle of long prompts.

Root cause: LLMs can struggle to focus on context placed in the middle of the input sequence.

Resolution: Rank and position the most relevant knowledge at prompt locations where the model attends better.

User language did not map cleanly to knowledge records.

Root cause: Everyday language was fragmented, grammatically incorrect, and varied.

Resolution: Use an LLM to translate each message into a coherent likely question and combine retrieval from both original and translated inputs.

Evaluation

GEICO evaluated the RAG/RagRails implementation through repeated sample tests focused on overpromising, LLM and developer effort, relevance checks, and whether retrieved contexts were applicable. The reported overpromising test improved from 12 incorrect responses out of 20, to 6, and eventually to none after RagRails adjustments.

Metrics: incorrect overpromising responses: 12 out of 20 before correction · incorrect responses reduced to 6 after initial RagRails-style adjustment · incorrect responses eventually reduced to none after further adjustments · LLM performance and developer effort for railed responses · retrieved-record relevance · Queries per Second protected by offline indexing

Outcomes

• GEICO created an experimental RAG implementation for conversational insurance interactions.
• The first RAG implementation using entire records was ineffective and was replaced with answer-only insertion.
• Input translation, ranking, and relevance checks were added to improve retrieval quality and context placement.
• RagRails reduced the overpromising test from 12 incorrect responses out of 20 to 6 and eventually to none.
• The team continued to explore RAG and other techniques rather than presenting the work as fully complete.

This case is a strong safety and reliability example rather than a mature production launch report. Its most reusable contribution is the RagRails idea: putting behavioral guidance into retrieved records so constraints appear only when relevant, avoiding global prompt bloat while reducing overpromising.