RAG-Powered Customer Support Enhancement Using GPT-4

thomson_reuters · Legal · 2024 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Legal and tax customer support knowledge retrieval

Thomson Reuters customer support agents needed faster, more accurate access to specialized legal and tax support knowledge spread across CRM systems, hundreds of thousands of knowledge-base articles, tickets, and person-to-person knowledge transfer.

Constraints:

• Support agents serve customers in specialized legal and tax domains where incorrect guidance is costly.
• Relevant information spans CRM tools, ticketing systems, and hundreds of thousands of knowledge-base articles.
• A standalone LLM response risks hallucination and lacks provenance.
• Knowledge changes over time, so the system needs an updateable non-parametric knowledge layer rather than only fixed model weights.

Architecture

Thomson Reuters implemented a RAG support assistant with two flows: a processing/indexing flow that chunks knowledge-base and CRM text, embeds it with sentence-transformer models, and stores it in Milvus; and a retrieval flow that embeds an agent query, retrieves semantically similar support passages, concatenates the selected context with the prompt, and sends it to GPT-4. The resulting chatty interface gives support agents product-specific answers grounded in Thomson Reuters support content.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["Knowledge base and CRM content"] --> B["Processing and indexing flow"]
  B --> C["Embedding model"]
  C --> D["Milvus vector database"]
  E["Customer support agent"] --> H["Support agent interface"]
  H --> F["Retrieval flow"]
  F --> C
  D --> F
  F --> G["GPT-4 API"]
  G --> H

Components

Data flow

Knowledge-base articles and CRM/ticket content are processed into chunks, converted into dense embeddings, and stored in Milvus. At query time, the customer-support agent asks a question, the query is embedded into the same vector space, dense retrieval returns relevant passages, those passages are concatenated with the prompt, and GPT-4 generates a support answer for the agent.

Why this architecture

Thomson Reuters chose RAG because support answers needed both the language capability of GPT-4 and grounding in a large, changing, domain-specific knowledge corpus. The non-parametric retrieval layer reduces hallucinations, provides provenance, and lets support knowledge update without retraining the base model.

1. Reduce hallucinations.

2. Provenance

3.More economical means to introduce new / update knowledge over retraining LLMs."

Models used: GPT-4 API · all-MiniLM-L6-v2 · BERT · RoBERTa · T5 · OpenAI text-embedding-ada-002

Scale

• The support corpus included hundreds of thousands of knowledge-base articles.
  “The customer support agents need to be able to quickly navigate CRM, hundreds of thousands of knowledge base articles, manage tickets”article, stated
• The company reports supporting customers all over the world.
  “Thomson Reuters is a leading business information and content technology provider, helping people all over the world make informed decisions that matter.”article, stated
• The solution targeted reduced resolution time for customer support.
  “we employed an AI-powered solution architecture known as RAG to deliver better, faster customer service. Read on to learn how we were able to reduce resolution times”article, stated

Alternatives considered (roads not taken)

Use GPT-4 without retrieval over Thomson Reuters support content

Rejected because: The article shows the non-RAG answer sounded plausible but was not the most accurate product-specific resolution.

Rely on parametric model knowledge alone

Rejected because: Fixed model knowledge is less adaptable when support data changes.

Continue person-to-person transfer of recent resolutions

Rejected because: Recent resolutions discovered by one agent were not structured for reuse by other agents.

Patterns

rag-retrieval (retrieval)

The system retrieves support passages from a dense vector database and feeds them to GPT-4 as grounded context.

retrieval-ranking (retrieval)

Dense retrieval ranks candidate passages by vector similarity to the agent's query.

human-in-the-loop-gate (workflow)

The generated answer is built for customer support agents, who use it in the customer support process rather than bypassing the agent.

fine-tuning-adaptation (model-adaptation)

The article explicitly chooses RAG's non-parametric adaptation path as a cheaper way to update knowledge than retraining the LLM.

Tradeoffs

Use RAG instead of raw GPT-4 for support answers.

Chose GPT-4 with retrieved support context over GPT-4 without RAG.

Rationale: The article's example shows the standalone response looked reasonable but missed the product-specific resolution.

Use non-parametric retrieval for changing knowledge.

Chose Flexible retrieval over post-training knowledge sources over Relying only on fixed parametric model knowledge.

Rationale: The support domain changes, so retrieval lets knowledge update after the base model is trained.

Use an open-source vector database and sentence-transformer stack for retrieval.

Chose Milvus and all-MiniLM-L6-v2 over Only using managed or proprietary retrieval components.

Rationale: The implementation names Milvus and a Hugging Face sentence transformer as the concrete retrieval stack.

Gotchas (what bit them)

Support agents suffered cognitive overload from scattered support systems.

Root cause: Agents had to navigate CRM, tickets, and hundreds of thousands of knowledge-base articles while serving specialized customers.

Resolution: Create a RAG interface that retrieves the most relevant solution from a curated support database.

Standalone GPT-4 could produce a plausible but wrong product answer.

Root cause: The model lacked the current, product-specific support resolution without retrieval context.

Resolution: Retrieve the product-specific resolution and feed it to GPT-4 with the prompt.

Support knowledge transfer was not structured for reuse.

Root cause: Recent resolutions could stay with one agent instead of becoming searchable organizational knowledge.

Resolution: Index support artifacts into dense retrieval so relevant passages can be reused by any agent query.

Evaluation

The article evaluates the RAG approach through a concrete before/after support question: GPT-4 without RAG produced a plausible generic tax answer, while GPT-4 with RAG returned the product-specific resolution steps.

Metrics: Resolution time reduction · Product-specific answer accuracy · With-RAG versus without-RAG answer comparison · Ability to provide provenance and reduce hallucinations

Outcomes

• The RAG assistant reduced support resolution times according to the article.
• The with-RAG example returned product-specific resolution steps where GPT-4 alone gave a generic answer.
• The architecture provided a path to update support knowledge without retraining the base LLM.
• Support agents gained a chatty interface over curated Thomson Reuters support knowledge.

The source reports a support corpus of hundreds of thousands of knowledge-base articles and global Thomson Reuters customers, but does not provide exact user counts, query volume, latency, or percentage reduction in resolution time.