Text-to-SQL Solution for Data Democratization in Food Delivery Operations

swiggy · E-commerce · 2024 · source Agentic: yes · promotional bias: med · summary-only (no fetched source) · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Food-delivery analytics and self-service text-to-SQL

Swiggy needed non-technical business users to access operational metrics through natural language instead of waiting on technical teams to write SQL, but the scale and variety of Swiggy's tables, columns, and business-specific context made a generic text-to-SQL approach insufficient.

Constraints:

The system had to serve food delivery and quick commerce business units.
Different business units had overlapping but distinct schemas and metrics.
Users accessed the system through Slack.
The pipeline had to return SQL and results quickly enough for decision-making.
The fetched source was unavailable, so this extraction is summary-only.

Architecture

Swiggy built Hermes, a Slack-based text-to-SQL system that evolved from a basic GPT-3.5 implementation to a RAG-based GPT-4o architecture. Hermes V2 compartmentalized business units into charters with dedicated metadata and knowledge bases, then used Databricks jobs, AWS Lambda middleware, Snowflake data, and Slack as the user interface.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  slack_interface["Slack interface"] --> aws_lambda_middleware["AWS Lambda middleware"]
  aws_lambda_middleware --> databricks_model_job["Databricks model job"]
  charter_knowledge_bases["Charter knowledge bases"] --> databricks_model_job
  react_column_retrieval_agent["ReAct column retrieval agent"] --> charter_knowledge_bases
  databricks_model_job --> snowflake_database["Snowflake database"]
  databricks_model_job --> aws_lambda_middleware

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Slack interface`	Natural-language entry point for business users.	`AWS Lambda middleware`
`AWS Lambda middleware`	Handles authentication, preprocessing, request routing, and response formatting.	`Slack interface`, `Databricks model job`
`Databricks model job`	Runs the charter-specific GPT-4o pipeline and SQL generation.	`AWS Lambda middleware`, `Charter knowledge bases`, `Snowflake database`
`Charter knowledge bases`	Business-unit-specific metadata and context for text-to-SQL generation.	`Databricks model job`
`Snowflake database`	Enterprise data warehouse queried by generated SQL.	`Databricks model job`
`ReAct column retrieval agent`	Planned or emerging agent to improve column retrieval and reduce prompt noise.	`Charter knowledge bases`, `Databricks model job`

Data flow

A Slack user asks a natural-language data question. Middleware formats and routes the request, Databricks launches the appropriate charter-specific GPT-4o job, the model uses charter metadata and knowledge bases to generate SQL, executes it against Snowflake, and returns the query plus results to Slack.

4 supporting quotes

“developed Hermes, a text-to-SQL solution that enables non-technical users to query company data using natural language through Slack.”summary, stated

“evolved from a basic GPT-3.5 implementation (V1) to a sophisticated RAG-based architecture with GPT-4o (V2)”summary, stated

“compartmentalizes business units into "charters" with dedicated metadata and knowledge bases.”summary, stated

“When a request arrives from middleware, a new Databricks job is created that fetches the appropriate charter's GenAI model (GPT-4o), generates the SQL query, executes it against the database”summary, stated

Why this architecture

Swiggy moved from a broad GPT-3.5 path to charter-specific RAG because a kitchen-sink prompt over all business data could not handle Swiggy's volume, schema complexity, or unit-specific context.

3 supporting quotes

“The "kitchen-sink approach" of treating all business needs and data uniformly proved ineffective.”summary, stated

“different business units had distinct but sometimes overlapping data structures—for example, metrics related to Swiggy Food differed from but resembled those for Swiggy Instamart.”summary, stated

“charters with well-defined metadata and relatively smaller numbers of tables performed substantially better”summary, stated

Models used: GPT-3.5 · GPT-4o · RAG · ReAct agent

Scale

Hermes served hundreds of users across the organization.
“Results include hundreds of users across the organization answering several thousand queries”summary, stated
The system answered several thousand queries.
“hundreds of users across the organization answering several thousand queries with average turnaround times under 2 minutes”summary, stated
Average turnaround was under 2 minutes.
“average turnaround times under 2 minutes”summary, stated

Alternatives considered (roads not taken)

Hermes V1 with basic GPT-3.5

Rejected because: It could not handle real Swiggy query complexity at scale.

“evolved from a basic GPT-3.5 implementation (V1) to a sophisticated RAG-based architecture with GPT-4o (V2)”summary, stated

Single kitchen-sink data context

Rejected because: Business-unit-specific data and metadata performed better when separated into charters.

“The "kitchen-sink approach" of treating all business needs and data uniformly proved ineffective.”summary, stated

Patterns

`rag-retrieval` (schema grounding)

Hermes V2 used charter-specific RAG metadata and knowledge bases to ground text-to-SQL.

“RAG-based architecture with GPT-4o (V2) that compartmentalizes business units into "charters" with dedicated metadata and knowledge bases.”summary, stated

`tool-use-orchestration` (data access)

The system orchestrates Slack, middleware, Databricks, GPT-4o, and Snowflake to answer user questions.

“The production system integrates multiple technologies: GPT-4o as the core language model, Databricks for compute and orchestration, AWS Lambda for middleware, Slack for the user interface, Snowflake as the data”summary, stated

`eval-driven-development` (quality)

The team evaluated V1 and V2 and found performance varied with charter metadata quality and table count.

“The team conducted systematic evaluations at both V1 and V2 stages. Initial assessments compared their implementation against multiple vendor solutions and research papers using out-of-the-box approaches with minimal Swiggy-specific modifications.”summary, stated

Tradeoffs

Compartmentalize business units into charters.

Chose Dedicated metadata and knowledge bases per charter over Uniform global context.

Rationale: Scoped context improved relevance and reduced confusion across similar but distinct business metrics.

“A particularly significant insight was that different business units had distinct but sometimes overlapping data structures—for example, metrics related to Swiggy Food differed from but resembled those for Swiggy Instamart.”summary, stated

Use Slack as the interface.

Chose Slack-based natural-language access over Separate BI or SQL-only tooling.

Rationale: The target users could ask questions without learning SQL or leaving their collaboration environment.

“enables non-technical users to query company data using natural language through Slack.”summary, stated

Gotchas (what bit them)

A single data context becomes noisy at enterprise scale.

Root cause: Swiggy had many tables, columns, and business-unit-specific meanings.

Resolution: Partition context into charters with dedicated metadata and knowledge bases.

“the sheer volume of data, tables, and columns that needed to be considered, and the necessity of incorporating Swiggy-specific business context.”summary, stated

Charter performance depends on metadata quality and table count.

Root cause: Text-to-SQL retrieval is more accurate when schema metadata is well-defined and scoped.

Resolution: Invest in charter-specific metadata and smaller scoped table sets.

“charters with well-defined metadata and relatively smaller numbers of tables performed substantially better”summary, stated

Evaluation

Swiggy compared V1 and V2, evaluated vendor/research approaches, and tracked charter-specific performance factors.

Metrics: Hundreds of users. · Several thousand queries. · Average turnaround under 2 minutes. · Charter-specific performance.

“Initial assessments compared their implementation against multiple vendor solutions and research papers using out-of-the-box approaches with minimal Swiggy-specific modifications.”summary, stated

“V2 evaluations revealed that performance varied significantly based on charter-specific factors”summary, stated

Outcomes

Hermes served hundreds of users.
It answered several thousand queries.
Average turnaround was under 2 minutes.
It reduced dependency on technical resources for business metrics.

The summary reports hundreds of users, several thousand queries, and average turnaround under 2 minutes.

“Results include hundreds of users across the organization answering several thousand queries with average turnaround times under 2 minutes”summary, stated