SQL Query Agent for Data Democratization

prosus · Tech · 2024 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Enterprise data access and natural-language SQL analytics

Prosus needed to let non-SQL specialists across a 30,000-employee portfolio ask operational data questions without waiting on human analysts, while keeping database scope, business context, query safety, and analyst oversight under control.

Constraints:

Different portfolio companies and domains require different databases, SQL dialects, and business definitions.
The system must avoid cross-domain data leakage when users ask questions in Slack.
Business context and schema descriptions must be explicit enough for an LLM to reason over metrics and abbreviations.
Generated queries need guardrails against expensive or dangerous SQL and a path to human escalation.

Architecture

Prosus built Token Data Analyst as a Slack-facing SQL agent. Each deployment is scoped to a use case, company, database, and SQL dialect, with routing handled by the Slack channel rather than by a single universal router. The agent uses specialized tools for table metadata, schema context, conversation history, SQL generation, and SQL execution, with a pre-generation clarity check to decide whether the question is sufficiently specified. Human analysts operate dev/prod instances, curate agent-ready context, monitor production responses, and handle escalations.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  business-user["Business user"] --> slack-channel["Slack channel"]
  slack-channel["Slack channel"] --> token-data-analyst-instance["Token Data Analyst instance"]
  token-data-analyst-instance["Token Data Analyst instance"] --> conversation-memory["Conversation memory"]
  token-data-analyst-instance["Token Data Analyst instance"] --> context-filtering-pipeline["Context filtering pipeline"]
  token-data-analyst-instance["Token Data Analyst instance"] --> clarity-check["Clarity check"]
  context-filtering-pipeline["Context filtering pipeline"] --> agent-ready-metadata["Agent-ready metadata"]
  clarity-check["Clarity check"] --> sql-generation-flow["SQL generation flow"]
  conversation-memory["Conversation memory"] --> sql-generation-flow["SQL generation flow"]
  context-filtering-pipeline["Context filtering pipeline"] --> sql-generation-flow["SQL generation flow"]
  sql-generation-flow["SQL generation flow"] --> sql-execution-tool["SQL execution tool"]
  sql-execution-tool["SQL execution tool"] --> query-guardrails["Query guardrails"]
  sql-execution-tool["SQL execution tool"] --> warehouse-connector["Warehouse connector"]
  warehouse-connector["Warehouse connector"] --> enterprise-data-warehouse["Enterprise data warehouse"]
  slack-channel["Slack channel"] --> human-analyst-monitor["Human analyst monitor"]
  human-analyst-monitor["Human analyst monitor"] --> agent-ready-metadata["Agent-ready metadata"]
  human-analyst-monitor["Human analyst monitor"] --> dev-instance["Dev instance"]
  dev-instance["Dev instance"] --> token-data-analyst-instance["Token Data Analyst instance"]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Slack channel`	Domain-specific user interface that routes questions to the relevant agent instance.	`Business user`, `Token Data Analyst instance`, `Human analyst monitor`
`Business user`	Non-SQL specialist asking operational data questions in natural language.	`Slack channel`
`Token Data Analyst instance`	Scoped SQL agent configured for a company, domain, database, and SQL dialect.	`Slack channel`, `Conversation memory`, `Context filtering pipeline`, `Clarity check`, `SQL generation flow`, `Dev instance`
`Conversation memory`	Maintains multi-turn context for the SQL assistant.	`Token Data Analyst instance`, `SQL generation flow`
`Context filtering pipeline`	Filters plain-text table, column, and business metadata to the subset needed for the user question.	`Token Data Analyst instance`, `Agent-ready metadata`, `SQL generation flow`
`Agent-ready metadata`	Curated table definitions, column descriptions, formulas, and abbreviations written for LLM use.	`Context filtering pipeline`, `Human analyst monitor`
`Clarity check`	Separate LLM call that detects underspecified questions before SQL generation.	`Token Data Analyst instance`, `SQL generation flow`, `Human analyst monitor`
`SQL generation flow`	Main agent reasoning path that creates SQL and passes it to the execution tool.	`Conversation memory`, `Context filtering pipeline`, `Clarity check`, `SQL execution tool`
`SQL execution tool`	Generalized query execution capability that can make small dialect-specific corrections.	`SQL generation flow`, `Warehouse connector`, `Query guardrails`
`Warehouse connector`	Infrastructure layer dispatching SQL to configured data stores such as Snowflake, Databricks, and BigQuery.	`SQL execution tool`, `Enterprise data warehouse`
`Enterprise data warehouse`	Connected data source for the specific use case or portfolio company.	`Warehouse connector`
`Query guardrails`	Safety controls that prevent expensive, destructive, or inappropriate database operations.	`SQL execution tool`
`Human analyst monitor`	Analysts who watch production channels, correct answers, curate context, and handle escalations.	`Slack channel`, `Agent-ready metadata`, `Clarity check`, `Dev instance`
`Dev instance`	Staged environment for analysts to test tables and context before production exposure.	`Token Data Analyst instance`, `Human analyst monitor`

Data flow

A business user asks a natural-language question in the relevant Slack channel. That channel maps to a domain-specific Token Data Analyst instance with configured company context, warehouse connector, and SQL dialect. The agent retrieves structured plain-text table and column context, runs a clarity check to decide whether it has enough information, generates SQL in the main reasoning flow, and sends the query to a generalized SQL execution tool. The infrastructure layer dispatches to Snowflake, Databricks, BigQuery, or another configured warehouse. Results return to Slack, where analysts can verify, correct, or escalate; dev instances let analysts test new tables and context before production rollout.

3 supporting quotes

“Routing is handled through dedicated Slack channels—users ask questions in the channel corresponding to their domain, and the appropriately configured agent responds.”summary, stated

“to the agents perspective is a generalized execute ability to execute SQL so it's an execute SQL Tool”transcript, stated

“before the agent attempts to generate SQL, a separate LLM call evaluates whether the question contains sufficient information to proceed.”summary, stated

Why this architecture

Prosus avoided a universal SQL agent and instead used scoped deployments because company, domain, warehouse, and dialect context mattered more than generic SQL syntax. General-purpose models were good enough at SQL generation, while the hard part was business-context reasoning and safe execution. Slack channels provided simple routing, human-visible oversight, and a natural escalation path; dev/prod instances supported staged rollout.

3 supporting quotes

“This design eliminates the complexity of having the agent determine which database to query and reduces the risk of cross-domain data leakage.”summary, stated

“they're really good at generating valid SQL you don't need a specialist at generating SQL they're very very good at that”transcript, stated

“agent responses appear in channels where human analysts are also present and can monitor, verify, and correct responses when needed.”summary, stated

Models used: OpenAI general-purpose models · LLM clarity-check call · LLM SQL reasoning agent

Scale

Prosus operates across about 30,000 employees.
“we're about 30,000 employees uh spread uh across various tech companies”transcript, stated
The system reduced time-to-answer by about 74%.
“we measured it's about 74% reduction um which is 74% reduction”transcript, stated
Connected datasets saw roughly a doubling or tripling of generated insights.
“so we have you know a doubling or tripling of the number of insights we call that that we can actually generate based on this data”transcript, stated
The agent connects to common enterprise stores including Snowflake, Databricks, and BigQuery.
“we can connect with Snowflake and with data breaks and with bigquery and many of the other um commonly used uh data stores”transcript, stated
Deployment spans multiple portfolio companies including iFood and OLX.
“now deployed across multiple portfolio companies including iFood and OLX.”summary, stated

Alternatives considered (roads not taken)

Single generalized agent that routes among all databases

Rejected because: Dedicated Slack channels and scoped agent instances avoided database-routing complexity and reduced leakage risk.

“Rather than building a single generalized agent that routes between databases, the team architected the system with separate agent instances per use case or database.”summary, stated

Fine-tuned text-to-SQL models

Rejected because: The team found general-purpose OpenAI models strong at SQL syntax and valued reasoning over business context more than SQL specialization.

“you don't need a specialized SQL agent for that you need a good Reasoner so it made sense to kind of stick with models that were good at reasoning”transcript, stated

Embedding-based RAG for schema retrieval

Rejected because: Schema embeddings were harder to maintain, harder to debug, and mismatched short user questions during iterative setup.

“Providing agents with appropriate database context proved to be the largest ongoing challenge. The team explicitly chose not to use RAG”summary, stated

One complex prompt that handles ambiguity inline

Rejected because: Prompting alone did not produce reliable uncertainty behavior, so they split uncertainty detection into a clarity check.

“Standard prompt engineering proved insufficient to instill appropriate uncertainty. The solution was to add a dedicated pre-processing step”summary, stated

Patterns

`tool-use-orchestration` (architecture)

The SQL agent uses specialized tools for context retrieval, SQL generation, execution, small syntax correction, and guarded database access.

“specialized tools handle specific tasks like schema retrieval and SQL execution, and conversation history maintains context for multi-turn interactions.”summary, stated

`context-engineering` (context-management)

Prosus invested in agent-ready metadata that spells out formulas, abbreviations, and assumptions as if onboarding a new analyst.

“The team learned to write context for agents as if onboarding a day-one data analyst: explicitly stating formulas, removing subjective language”summary, stated

`prompt-chaining` (architecture)

A separate clarity-check LLM call screens for ambiguity before the main SQL-generation flow proceeds.

“This isolated task framing helped the model behave more appropriately. The team built a "clarity dataset" specifically for evaluating model performance on self-doubt—testing whether models can recognize when they lack adequate context.”summary, stated

`human-in-the-loop-gate` (governance)

Human analysts remain in Slack channels and dev environments to verify, correct, escalate, and promote context changes.

“agent responses appear in channels where human analysts are also present and can monitor, verify, and correct responses when needed.”summary, stated

`safety-policy-enforcement` (governance)

Query guardrails prevent dangerous or expensive SQL such as broad billion-row scans and DDL operations.

“Guardrails were essential to prevent expensive or dangerous queries (SELECT * on billion-row tables, inadvertent DDL operations).”summary, stated

Tradeoffs

Route by Slack channel and scoped agent instance

Chose Separate domain/database-specific agents over One generalized router agent.

Rationale: The scoped design removes database-choice ambiguity and limits cross-domain exposure.

“Each instance is configured with a specific SQL dialect (PostgreSQL, T-SQL, etc.) and connects to a particular data source”summary, stated

Use general-purpose reasoning models

Chose OpenAI models with business-context reasoning over Fine-tuned text-to-SQL models.

Rationale: The team judged SQL syntax generation less difficult than reasoning over business context and table metadata.

“what they're not very good at is reasoning over the context around the table metadata and you don't need a specialized SQL agent for that you need a good Reasoner”transcript, stated

Use plain-text filtered context

Chose Structured metadata filtering over Embedding-based schema retrieval.

Rationale: Plain text was easier to maintain and debug while schemas and analyst context changed frequently.

“• Debugging difficulty: Plain text reasoning is far easier to diagnose than embedding-based retrieval failures”summary, stated

Keep analysts in the production loop

Chose Augmentation with visible human monitoring over Fully autonomous self-service analytics.

Rationale: The agent targets repeatable middle-ground questions while complex analysis and trust maintenance still require analysts.

“The Token Data Analyst was designed as an augmentation to existing data analyst teams rather than a replacement.”summary, stated

Gotchas (what bit them)

Correct-looking SQL answers can be the scariest failure mode.

Root cause: When questions are underspecified, the model may assume context and produce plausible but incorrect results.

Resolution: Add a clarity-check step and a clarity dataset to evaluate whether the model recognizes insufficient context.

“This represented the "scariest failure case" because responses appeared legitimate to users unfamiliar with the underlying data.”summary, stated

Human-authored metadata is not automatically agent-ready.

Root cause: Analyst documentation often relies on implied formulas, abbreviations, or business assumptions.

Resolution: Rewrite metadata as day-one analyst onboarding material with explicit formulas and definitions.

“Any statement with room for interpretation would be interpreted unpredictably by the model. This represents a significant ongoing investment in context curation, which remains the primary bottleneck for onboarding new use cases.”summary, stated

Retry loops appeared in early versions.

Root cause: The agent could keep trying to repair or rerun queries without graceful failure boundaries.

Resolution: Add circuit-breaker logic and make the SQL execution tool capable of small corrections without full agent reruns.

“• Loop detection: Early versions could get stuck in retry loops; circuit-breaker logic was needed to force graceful failure.”summary, stated

Model upgrades changed behavior enough to require rollback and retuning.

Root cause: Prompts and context became overfit to specific OpenAI model versions.

Resolution: Treat model version changes as a validation event and continue work toward more model-agnostic context.

“Upgrading to newer OpenAI models caused significant behavior changes, requiring rollbacks and re-optimization.”summary, stated

Evaluation

Prosus evaluated the agent through business-impact metrics, clarity-check datasets, analyst dev/prod testing, and production monitoring in Slack channels. They measured time-to-answer and usage/insight volume after connecting specific datasets.

Metrics: time-to-answer reduction · number of insights generated · query volume against connected datasets · clarity-check performance · production analyst corrections and escalations

“we're only measuring here the actual time it takes for somebody to go and once they're already working it”transcript, stated

“The team built a "clarity dataset" specifically for evaluating model performance on self-doubt—testing whether models can recognize when they lack adequate context.”summary, stated

Outcomes

Prosus reports a 74% reduction in time-to-answer for data queries.
Connected datasets saw a doubling or tripling of generated insights because more users could ask questions directly.
The system expanded across multiple portfolio companies, including iFood and OLX.
Human analysts remained responsible for complex analytical work and production oversight.

The case cites roughly 30,000 employees, thousands of analysts, 74% time-to-answer reduction, 2-3x insight growth, and integrations with Snowflake, Databricks, and BigQuery.

“Results include a 74% reduction in time-to-answer for data queries and a doubling or tripling of insights generated due to reduced bottlenecks”summary, stated

“Results include a 74% reduction in time-to-answer for data queries and a doubling or tripling of insights generated due to reduced bottlenecks, with the system now deployed across multiple portfolio companies including iFood and OLX.”summary, stated