Building a Production-Ready Business Analytics Assistant with ChatGPT

microsoft · Tech · 2023 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Business analytics assistant over structured SQL data

Microsoft's reference implementation explored how to make ChatGPT answer complex analytical questions over structured business data despite LLM weaknesses in quantitative reasoning, schema selection, output consistency, hallucination, and token limits.

Constraints:

• LLMs are less reliable for quantitative reasoning and numeric work
• Business analytics depends on structured SQL data and accurate calculations
• Users may lack advanced technical skills but still need rich analytical answers
• Large database schemas can exceed token limits if loaded wholesale
• Generative output formats can be inconsistent
• Hallucination and randomness require validation and user-visible intermediate outputs

Architecture

The design uses ChatGPT as the planning brain and external tools for the actual data work. A Data Scientist agent interacts with the user, plans the analysis with ReAct, asks a Data Engineer agent to retrieve source data from SQL systems, executes Python analytics and visualization code, and presents results through a Streamlit application. The platform wraps SQL, Python, Plotly, display, persistence, and session-state utilities so the agents can use simple tool actions rather than direct complex APIs.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  user[User] --> streamlit-application[Streamlit Application]
  streamlit-application --> data-scientist-agent[Data Scientist Agent]
  streamlit-application --> session-state[Session State]
  data-scientist-agent --> data-engineer-agent[Data Engineer Agent]
  data-engineer-agent --> schema-retrieval-tool[Schema Retrieval Tool]
  schema-retrieval-tool --> sql-database[SQL Database]
  data-engineer-agent --> sql-query-tool[SQL Query Tool]
  sql-query-tool --> sql-database
  sql-query-tool --> python-console[Python Console]
  data-scientist-agent --> python-console
  python-console --> analytics-libraries[Analytics Libraries]
  analytics-libraries --> plotly-visualization[Plotly Visualization]
  plotly-visualization --> display-utility[Display Utility]
  display-utility --> streamlit-application

Components

Data flow

A user asks a business question in the application. The Data Scientist agent plans the analysis, requests data from the Data Engineer agent, the Data Engineer identifies needed tables, retrieves schemas, writes SQL, and returns data. The Data Scientist then runs Python analytics or ML code, generates Plotly visualizations, and displays final results in Streamlit while ReAct observations let both agents adjust intermediate steps.

Why this architecture

The architecture separates high-level reasoning from numeric execution. ChatGPT plans, decomposes, and writes tool calls, while SQL, Python, analytics libraries, and Plotly do the reliable data access, computation, and visualization. Two agents split database acquisition from analytical reasoning, reducing task complexity and giving each agent a narrower prompt and tool surface.

Models used: ChatGPT

Scale

• The implementation supports two SQL source options, SQLite and SQL Server.
  “SQL or structured data is the primary source for data analysis. In this implementation, two options are provided — SQLite and SQL Server — but you can easily expand them to any SQL source with only minor changes.”article, stated
• The production suggestion separates agents as RESTful backend APIs from a Streamlit frontend.
  “you may choose to deploy the agents as restful back-end APIs using a framework such as Flask and deploy the UI layer using Streamlit”article, stated
• The dynamic context strategy uses three stages: identify tables, retrieve schema, then build a query.
  “In this implementation, this is done in three steps: 1.) Identify the tables needed for the question, 2.) Retrieve the detail schema of the identified tables, and 3.) Build a data retrieval query based on the schema and the user’s request.”article, stated

Alternatives considered (roads not taken)

Have ChatGPT perform calculations directly

Rejected because: LLMs are less reliable for quantitative reasoning, so the design makes tools perform calculations and data retrieval.

Use one general-purpose agent for all work

Rejected because: Splitting data acquisition from analysis made the problem more tractable for ChatGPT.

Load the full database schema into every prompt

Rejected because: Large schemas can exceed token limits, so context is built dynamically around the needed tables.

Expose complex APIs directly to the model

Rejected because: Complex tools confuse ChatGPT and consume few-shot budget, so the implementation wraps them behind simpler APIs.

Patterns

multi-agent-orchestration (agent-architecture)

A Data Scientist agent delegates source-data acquisition to a Data Engineer agent, then performs analysis and presentation.

tool-use-orchestration (analytics)

ChatGPT uses SQL, Python, data-science libraries, Plotly, and display utilities as tools rather than doing the work internally.

prompt-chaining (reasoning)

Both agents use ReAct thought/action/observation loops to plan, execute, observe, and adjust.

structured-output-contract (reliability)

The implementation wraps tools and uses validation/retry flows to handle format inconsistency and unreliable outputs.

human-in-the-loop-gate (analytics-validation)

The application should expose intermediate SQL/code outputs so users can validate model-generated analytical steps.

Tradeoffs

Use ChatGPT as an orchestrator, not a calculator

Chose LLM planning plus SQL/Python execution over direct LLM quantitative reasoning.

Rationale: This preserves ChatGPT's high-level reasoning strengths while moving computations into deterministic tools.

Split into Data Engineer and Data Scientist agents

Chose two specialized ChatGPT agents over one agent handling data access and analysis end to end.

Rationale: The split isolates database complexity from analytical reasoning and makes each subtask easier for ChatGPT.

Use Streamlit as the application platform

Chose Streamlit over Dash, Bokeh, or a more custom UI stack.

Rationale: Streamlit provided stateful memory and simpler development for an interactive analytics assistant.

Dynamically build database context

Chose table identification, schema retrieval, and query construction over static full-schema prompting.

Rationale: Dynamic context avoids token limits and narrows the prompt to the question-specific schema.

Gotchas (what bit them)

LLMs are unreliable at quantitative reasoning when left unaided.

Root cause: ChatGPT is trained primarily around language and communication rather than numeric computation.

Resolution: The system delegates data retrieval and calculation to SQL, Python, and analytics tools.

Database schemas can overflow the prompt.

Root cause: Analytics sources may contain many tables and objects, and loading the full schema can exceed token limits.

Resolution: Build context dynamically by selecting tables, retrieving only needed schemas, and then constructing SQL.

Output format inconsistency needs production handling.

Root cause: Generative models retain randomness even with clear instructions.

Resolution: Use validation and retry flows around outputs.

Complex tools confuse agents and consume prompt budget.

Root cause: Complex APIs and interaction flows require many examples for the model to learn.

Resolution: Wrap APIs behind simple utility functions before exposing them to ChatGPT.

Hallucination remains a reliability risk for analytics.

Root cause: LLM randomness can affect accuracy even when tools are available.

Resolution: The design calls for multiple validation mechanisms and visible intermediate SQL/code.

Evaluation

The article frames evaluation as production-hardening work rather than benchmarked deployment results: validate and retry inconsistent outputs, show intermediate SQL/code for user inspection, reduce tool complexity, and use multiple validation mechanisms at intermediate processing steps.

Metrics: output format validity · SQL query correctness · Python code correctness · intermediate-step validation · token budget · visualization correctness · user inspection of generated SQL and code

Outcomes

• The case study produced a reference methodology and implementation for a ChatGPT-powered business analytics assistant.
• The architecture uses separate Data Engineer and Data Scientist agents coordinated through ReAct loops.
• The implementation exposes SQL, Python, analytics, Plotly, display, and state utilities as tools.
• The author identifies production risks around schema context, output consistency, tool complexity, hallucination, and validation.
• The design can be deployed with backend agent APIs and a Streamlit UI for production-style separation.

This was a reference implementation rather than a reported production deployment. Its concrete scale claims are architectural: support for SQLite and SQL Server, a three-step dynamic schema-context strategy, and optional production deployment as REST backends plus Streamlit frontend.