Natural Language Interface to Business Intelligence Using RAG

volvo · Automotive · 2024 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Operational business intelligence

Volvo Car Mobility needed to reduce interruptions to data analysts by letting business stakeholders ask operational metrics questions directly in Slack.

Constraints:

Architecture

An AI engineer built a Slack bot that sends stakeholder questions plus the full database schema and DBT metadata to GPT-4 Turbo, generates BigQuery SQL, and returns results in Slack.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  slack-bot["Slack Bot"]
  gpt-4-turbo-query-generator["GPT-4 Turbo Query Generator"]
  schema-and-dbt-context["Schema and DBT Context"]
  bigquery["BigQuery"]
  slack-bot --> gpt-4-turbo-query-generator
  gpt-4-turbo-query-generator --> schema-and-dbt-context
  gpt-4-turbo-query-generator --> bigquery
  schema-and-dbt-context --> gpt-4-turbo-query-generator
  bigquery --> slack-bot
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Slack Bot Receives business questions and returns query results. GPT-4 Turbo Query Generator
GPT-4 Turbo Query Generator Translates natural-language questions into BigQuery SQL using schema context. Schema and DBT Context, BigQuery
Schema and DBT Context Provides database structure and semantic metadata to the model. GPT-4 Turbo Query Generator
BigQuery Executes generated SQL against operational metrics data. Slack Bot

Data flow

A stakeholder asks a question in Slack, the bot passes the question with schema and DBT metadata into GPT-4 Turbo, the model generates BigQuery SQL, and the bot returns query results to Slack.

“built a Slack bot that uses ChatGPT to translate natural language questions into BigQuery SQL queries, returning results directly in Slack.”summary, stated
“include the entire database schema and DBT metadata as context, eliminating the need for vector database semantic search.”summary, stated

Why this architecture

The expanded GPT-4 Turbo context window let the team keep schema grounding in prompt context rather than operating a separate vector-search layer.

“leverages the expanded context window of GPT-4 Turbo (128K tokens) to include the entire database schema and DBT metadata as context”summary, stated
“eliminating the need for vector database semantic search.”summary, stated

Models used: GPT-4 Turbo · ChatGPT · BigQuery

Scale

Alternatives considered (roads not taken)

Vector database semantic search over schema metadata

Rejected because: The full schema and DBT metadata fit into GPT-4 Turbo's expanded context window.

“eliminating the need for vector database semantic search.”summary, stated

Continue routing ad-hoc questions to analysts

Rejected because: Analysts were frequently interrupted by Slack questions.

“data analysts being frequently interrupted by business stakeholders asking ad-hoc questions about operational metrics via Slack.”summary, stated

Patterns

query-rewriting (query-generation)

The bot rewrites natural-language Slack questions into BigQuery SQL.

“translate natural language questions into BigQuery SQL queries”summary, stated

context-engineering (context)

The entire schema and DBT metadata are curated into the model context instead of retrieved separately.

“include the entire database schema and DBT metadata as context”summary, stated

tool-use-orchestration (tooling)

The Slack bot coordinates LLM SQL generation and BigQuery execution.

“returning results directly in Slack.”summary, implied

Tradeoffs

Put full schema context in the prompt

Chose GPT-4 Turbo 128K context over Vector database semantic search.

Rationale: The large context window simplified the architecture by removing a retrieval layer.

“include the entire database schema and DBT metadata as context, eliminating the need for vector database semantic search.”summary, stated

Deliver BI in Slack

Chose Slack bot results over Separate analytics portal workflow.

Rationale: Stakeholders were already asking operational metric questions in Slack.

“returning results directly in Slack.”summary, stated

Gotchas (what bit them)

Analyst interruptions can become the bottleneck in operational BI.

Root cause: Business users defaulted to asking analysts ad-hoc Slack questions.

Resolution: Move routine questions into a Slack-native natural-language query bot.

“data analysts being frequently interrupted by business stakeholders asking ad-hoc questions about operational metrics via Slack.”summary, stated

Schema grounding architecture depends on context-window size.

Root cause: The design avoids vector search only because the schema and metadata fit in the prompt.

Resolution: Use GPT-4 Turbo's 128K context window for the full schema and DBT metadata.

“expanded context window of GPT-4 Turbo (128K tokens)”summary, stated

Evaluation

The summary reports production use and analyst interruption relief, but no accuracy or latency benchmark.

Metrics: Production usage · Routine query automation

“The system is now in production, automating routine data queries and freeing up the data team for higher-value work.”summary, stated

Outcomes

The report gives a 128K context-window design detail but no query volume.

“The system is now in production, automating routine data queries and freeing up the data team for higher-value work.”summary, stated