RAG-Powered LLM System for Automated Analytics and Fraud Investigation

grab · Tech · 2024 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Analytics automation and fraud investigation

Grab's Integrity Analytics team needed to reduce repetitive analyst work for report generation and fraud investigations, where analysts were repeatedly writing similar SQL queries and manually assembling insights for stakeholders.

Constraints:

• The system needed real-time or near-real-time standardized data to keep LLM outputs accurate.
• Analysts already knew the exact SQL queries or functions needed for many tasks, so the LLM needed controlled access to those tools rather than free-form guessing.
• Outputs needed to reach users through existing collaboration channels such as Slack.
• The team wanted to avoid the cost and update burden of fine-tuning whenever production data changed.

Architecture

Grab built a RAG-powered analytics automation system around Spellvault, Data-Arks, scheduled execution, and Slack delivery. Spellvault provides low/no-code RAG and prompt management. Data-Arks packages frequently used SQL queries and Python functions as APIs, including integrations with Slack, Wiki, and JIRA. Report summarizers and the A* fraud investigation bot use Spellvault to select relevant Data-Arks APIs, execute them, summarize the returned structured data, and deliver results through Slack.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["Data analyst queries"] --> B["Data-Arks"]
  H["Wiki"] --> B
  I["JIRA"] --> B
  B --> C["Spellvault"]
  C --> D["Scheduler"]
  D --> E["Report Summarizer"]
  E --> B
  E --> G["Slack"]
  C --> F["A* bot"]
  F --> B
  F --> G
  B --> G

Components

Data flow

Analysts package repeatable SQL queries and Python functions into Data-Arks APIs. A scheduled report or user prompt reaches Spellvault, which uses RAG to select the relevant API or query, calls Data-Arks to retrieve standardized near-real-time data, and passes that data to the LLM for summarization. The generated report or fraud-investigation summary is delivered through Slack.

Why this architecture

Grab did not use an LLM as a standalone analyst. It wrapped the LLM with a controlled API layer for exact SQL and Python functions, because the team already knew the data and format needed for many tasks. RAG beat fine-tuning because it could retrieve current production data, reuse one base model across use cases, and scale new applications without retraining.

Models used: GPT · GPT-4o

Scale

• Grab operates across 700 cities in eight countries.
  “Grab offers a wide range of on-demand services in the region, including mobility, food, package and grocery delivery services, mobile payments, and financial services across 700 cities in eight countries.”article, stated
• Automated report generation saved an estimated 3-4 hours per report.
  “This automated report generation has helped save an estimated 3-4 hours per report.”article, stated
• Fraud investigations that were time-consuming were reduced to minutes.
  “what was once a time-consuming fraud investigation can now be reduced to a matter of minutes”article, stated

Alternatives considered (roads not taken)

Manual analyst query writing and report summarization

Rejected because: Repeating similar SQL queries was time-consuming and inefficient for analysts.

Fine-tune models for each analytical use case

Rejected because: Fine-tuning would add compute cost and require retraining for new information.

Let the LLM infer data directly without controlled APIs

Rejected because: The system needed exact SQL execution and standardized data inputs for accurate outputs.

Patterns

tool-use-orchestration (orchestration)

The LLM workflow calls Data-Arks APIs that package SQL queries, Python functions, and collaboration-tool integrations.

rag-retrieval (retrieval)

Spellvault uses RAG to select relevant queries and context for report generation and fraud investigation.

batch-offline-processing (workflow)

Scheduled LLM applications generate routine report summaries at regular intervals.

fine-tuning-adaptation (model-adaptation)

The team explicitly chose RAG instead of fine-tuning as the adaptation mechanism for current data and cost efficiency.

Tradeoffs

Use RAG rather than fine-tuning for analytics accuracy.

Chose RAG over production data and managed knowledge bases over Fine-tuning on domain-specific data.

Rationale: RAG was cheaper, more flexible across use cases, and able to use current production data without retraining.

Package known analyst work as APIs.

Chose Data-Arks APIs for SQL queries and Python functions over Manual analyst execution or unconstrained LLM data access.

Rationale: The team often knew the required data and format, so exact API execution reduced ambiguity.

Deliver insights through Slack.

Chose Slack as the report and investigation interface over A separate bespoke analytics UI.

Rationale: The article frames messaging as the user interface for commands, reports, and insights.

Gotchas (what bit them)

LLMs alone were only a fragment of the analytics automation system.

Root cause: Analytical tasks require tools for data retrieval, scheduling, and user interaction around the model.

Resolution: Integrate Spellvault with Data-Arks, a scheduler, and Slack to form a complete workflow.

Data format and latency directly affected answer accuracy.

Root cause: The LLM depends on standardized, current data inputs.

Resolution: Use Data-Arks to serve exact query outputs in a standardized format with real-time or near-real-time data.

Fine-tuning would make fresh production-data answers expensive to maintain.

Root cause: A fine-tuned model would need retraining for each new information update.

Resolution: Use RAG over production databases and knowledge bases so current context can be retrieved at answer time.

Evaluation

The article demonstrates results through sample report and fraud-investigation outputs, and it frames operational success in time savings: 3-4 hours saved per report and fraud investigations reduced to minutes.

Metrics: Estimated 3-4 hours saved per report · Fraud investigations reduced to minutes · Ability to provide necessary information simultaneously · Current-data responses without retraining

Outcomes

• Report generation saved an estimated 3-4 hours per report.
• Fraud investigations could be reduced to minutes.
• Employees across teams could self-serve Data-Arks APIs for their use cases.
• The architecture set Grab up to add more integrations and multimodal GPT-4o use cases.

Grab's broader operating context is 700 cities in eight countries; the article reports report-level and investigation-level time reductions but not query volume or active user counts for this specific system.