Building Kepler: An AI Data Analyst Agent for Internal Data Exploration

openai · Tech · 2026 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Internal data analyst agent

OpenAI needed an internal agent that could answer data questions across a huge, fast-growing data platform without requiring users to manually discover tables, decode schema nuance, write long SQL, chase Slack/Notion context, and validate assumptions themselves.

Constraints:

• The data platform served 80% of the company across 15 tools, over 600 petabytes of data a day, and 70k datasets.
• Table discovery was difficult because many tables sounded similar but differed in encryption, filtering, fraud adjustment, lineage, and semantics.
• A single missed nuance could make an answer wrong by an order of magnitude.
• All table schemas and query history could not fit into a model context window.
• The agent had to respect existing user permissions and avoid leaking sensitive IDs or PII, especially in Slack.
• SQL generation could not be evaluated by string equality because equivalent SQL can be written many ways.

Architecture

Kepler is an AI data analyst service that connects Slack, web UI, IDE, and MCP entry points to an LLM armed with a data-platform toolbox. Offline jobs build retrievable knowledge bases from table schemas, query history, lineage, code-crawled Codex descriptions, and company context from Slack, Notion, and Google Drive. At runtime, Kepler retrieves relevant context, uses MCP tools to search knowledge, inspect schemas, run data warehouse/Spark/Airflow calls, iterate over query results, and produce SQL, charts, anomaly explanations, or table answers. Memory stores corrections at user, channel, and global scopes. Evals compare generated and expected SQL/results with AST normalization and an OpenAI model grader. Security passes through user auth, redacts broad Slack outputs, and links raw results through permission-checked external UIs.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  data-question-user[Data question user] --> kepler-entry-points[Kepler entry points]
  kepler-entry-points --> kepler-agent-service[Kepler agent service]
  kepler-agent-service --> llm-reasoning-model[LLM reasoning model]
  kepler-agent-service --> mcp-toolbox[MCP toolbox]
  kepler-agent-service --> retrieved-context-layer[Retrieved context layer]
  kepler-agent-service --> memory-store[Memory store]
  kepler-agent-service --> raw-results-ui[Raw results UI]
  mcp-toolbox --> data-warehouse-and-platform-apis[Data warehouse and platform APIs]
  mcp-toolbox --> retrieved-context-layer
  offline-table-context-jobs[Offline table context jobs] --> retrieved-context-layer
  offline-table-context-jobs --> codex-code-crawling-tasks[Codex code crawling tasks]
  company-knowledge-service[Company knowledge service] --> retrieved-context-layer
  memory-suggestion-flow[Memory suggestion flow] --> memory-store
  memory-suggestion-flow --> data-question-user
  evaluation-system[Evaluation system] --> kepler-agent-service
  evaluation-system --> openai-evals-grader[OpenAI evals grader]
  authorization-and-redaction-layer[Authorization and redaction layer] --> kepler-agent-service
  authorization-and-redaction-layer --> raw-results-ui
  raw-results-ui --> data-question-user

Components

Data flow

A user asks Kepler a data question through Slack, UI, Cursor/MCP, or another MCP platform. Kepler retrieves table metadata, code-generated table descriptions, lineage, query history, company docs, and scoped memories from embedded knowledge bases. It then uses MCP tools to search, inspect schemas, run warehouse/Spark/Airflow calls, examine results, revise bad joins or thresholds, and continue until it has a defensible answer. When data is returned, Kepler provides linked query artifacts and raw-result references. The eval system separately sends natural-language questions through the agent endpoint, runs generated and expected SQL, compares result sets, and asks an OpenAI evals grader to score semantic correctness.

Why this architecture

Kepler's architecture exists because data-analysis correctness depends on context, iteration, and permissioned live access. Offline retrieval makes 70k tables and company context searchable without stuffing everything into a prompt. MCP gives the agent a recoverable tool loop for schema checks, queries, and warehouse calls. Memory captures hard-to-infer local corrections. Evals protect against regressions in SQL generation. Auth pass-through and redaction let the agent act as an analyst without becoming a new entitlement surface.

Models used: OpenAI models · GPT-5 · OpenAI embedding API · OpenAI evals grader · Codex

Scale

• 80% of OpenAI directly used the data platform.
  “At OpenAI, 80% of the company directly uses our data platform.”article, stated
• The platform had 15 tools processing over 600 petabytes of data per day.
  “That's 80% of the company using our team's 15 tools to process over 600 petabytes of data a day across 70k total datasets.”article, stated
• Kepler had to reason over about 70k total datasets.
  “across 70k total datasets. The data is just growing even more rapidly.”article, stated
• Manual resolution of one simple question involved three code deep dives, two meetings, and five Slack threads.
  “After three code deep dives, two quick meetings, and five Slack threads, we finally have an answer.”article, stated
• A representative SQL statement reached 160 lines of code.
  “This SQL statement is 160 lines of code, and I have no idea if it's wrong or not.”article, stated
• One wrong-answer example had 5 million ChatGPT users instead of the actual 800 million.
  “the agent thought that there were 5 million ChatGPT users compared to the actual 800 million answer.”article, stated
• Kepler supports 24/7 use and long follow-up threads, including a 56-reply thread.
  “Kepler is available 24/7 for a quick conversation. You can also do follow-ups. That's how you get a thread with 56 replies”article, stated
• The taxi demo returned 375 total pickups for a follow-up date query.
  “get the results that there were 375 total pickups in this dataset.”article, stated
• The memory system had three scopes: user, channel, and global.
  “We have three scopes right now. There are user level ones because users might want their own customizations.”article, stated

  “There's channel level for team scoped memories, and then also global memories for just general fixes that can benefit everyone.”article, stated
• Context alone was estimated to get the agent 80-90% of the way to the right answer before memory corrections.
  “Context will get you maybe 80%, 90% of the way there. Sometimes you need those final little corrections”article, stated

Alternatives considered (roads not taken)

Continue manual data-question escalation through data scientists, engineers, meetings, and Slack.

Rejected because: Simple questions were too slow and expensive to answer manually at OpenAI's data scale.

Put all table schemas and query history into the model context.

Rejected because: 70k tables made that impossible, so Kepler preprocesses and retrieves relevant context instead.

Rely on table schemas alone.

Rejected because: Schemas lacked semantics, relationships, lineage, query history, freshness, grain, primary keys, and alternate-table guidance needed for correct answers.

Evaluate generated SQL by exact text equality.

Rejected because: Semantically equivalent SQL can differ syntactically, so the eval normalizes ASTs, compares result sets with tolerance, and uses a model grader.

Give Kepler extra authorization over data.

Rejected because: The system passes through user authentication so it cannot grant tables the user lacks, while still linking authorized users to raw result views.

Start from a web-first agent surface only.

Rejected because: OpenAI's analytics work already happened in Slack, so a Slack interface drove adoption by meeting users where they asked questions.

Patterns

tool-use-orchestration (agent-architecture)

Kepler uses MCP tools to search knowledge, inspect schemas, run queries, examine results, and recover when an intermediate action such as a join key is wrong.

rag-retrieval (retrieval)

Kepler retrieves embedded table metadata, query history, lineage, code-generated descriptions, Slack threads, Notion docs, and Google Drive chunks at runtime.

agent-memory (state-management)

Kepler stores corrections and learnings as retrievable memories scoped to users, channels, and global fixes, with suggestions, edits, pruning, and runtime retrieval.

eval-driven-development (evaluation)

Kepler's team built question/expected-SQL evals, generated SQL through the agent endpoint, ran expected and generated queries, and used a model grader with AST-normalized SQL and result comparison.

observability-tracing (observability)

Kepler streams chain-of-thought for audit, user interruption, debugging assumptions, and eval failure diagnosis.

safety-policy-enforcement (security)

Kepler passes through user authentication, avoids granting extra access, redacts broad Slack outputs, ingests pre-sanitized queries, and gates raw results by table permissions.

data-quality-loop (operations)

User feedback, Slack threads, manual thumbs up/down, common questions, memory edits, and eval questions feed back into prompt, backend, memory, eval, and future fine-tuning work.

Tradeoffs

Use retrieval and preprocessing instead of stuffing all data context into prompts.

Chose embedded table/company context plus live retrieval over full schema/query-history context in every model call.

Rationale: The table universe was too large for context windows, and retrieval let Kepler load just the relevant table, lineage, query, company, and memory context at runtime.

Autogenerate rich table descriptions with Codex tasks.

Chose daily parallel code-crawling jobs over manual data-catalog descriptions alone.

Rationale: Manual descriptions go stale and miss creation, lineage, downstream usage, grain, and freshness details that determine correct analysis.

Expose chain-of-thought and query artifacts.

Chose streamed reasoning, links to queries, and reference IDs over opaque final answers only.

Rationale: Users need to audit assumptions, interrupt wrong paths, inspect raw results, and debug eval failures in a high-stakes data-analysis setting.

Grade SQL semantically rather than syntactically.

Chose AST normalization, result-set wiggle room, and LLM-based grading over exact SQL text equality.

Rationale: Equivalent SQL often differs in syntax or harmless result details, while a model grader can reason about whether the generated answer actually matches the expected result.

Preserve existing permissions instead of giving the agent broad data access.

Chose pass-through authentication and permission-checked raw result links over agent-specific superuser access to tables and raw results.

Rationale: Kepler answers questions without becoming a privilege escalation layer, while still giving authorized users access to raw artifacts they could have produced themselves.

Meet users in Slack before relying on a web interface.

Chose Slack-first agent adoption over web-only analytics assistant UX.

Rationale: OpenAI users already asked analytics questions and posted updates in Slack, so a Slack agent lowered adoption friction.

Gotchas (what bit them)

Agents without the right data context can produce wildly wrong answers.

Root cause: Similar table names, hidden schema differences, stale descriptions, and missing company context make the model infer the wrong source or meaning.

Resolution: Retrieve rich table metadata, query history, lineage, company context, code-crawled descriptions, and memories before and during analysis.

Small table-semantics mistakes can create catastrophic business answers.

Root cause: Tables can differ by encryption, fraud adjustment, pre-filtering, feedback status, or other nuances that are not obvious from names alone.

Resolution: Autogenerate richer table descriptions from code, lineage, query history, grain, primary keys, and freshness rather than depending on names or schemas alone.

Too many overlapping tools can confuse the model.

Root cause: Similar tool calls with subtle distinctions such as service auth versus user auth made tool discovery harder for the agent.

Resolution: Reduce and simplify the available tool calls so the model can find the right tool more reliably.

Overly specific prompts made the agent worse.

Root cause: Data questions branch in many ways, so rigid instructions forced the model down paths that did not fit the user's actual question.

Resolution: Use more general prompting and let GPT-5 reasoning choose the path from the available context and tools.

Memory can regress answers if bad or conflicting memories are retrieved.

Root cause: Generated memories may be accidental, non-reusable, scoped poorly, or simply wrong.

Resolution: Scope memories, let users inspect/edit them, prune them offline, and use chain-of-thought/auditing when bad memories influence answers.

Trust depends on validation beyond a plausible answer.

Root cause: Kepler can make mistakes and users will not adopt it if answers cannot be audited or validated against known sources.

Resolution: Stream assumptions, link queries/raw results, run evals, and plan extra validation steps such as comparing results against dashboard numbers.

Evaluation

Kepler uses both formal evals and live user feedback. Formal evals are question-answer pairs with manually curated expected SQL. The system generates SQL through Kepler, runs generated and expected queries, normalizes SQL to ASTs, compares result sets with semantic tolerance, and asks an OpenAI evals grader to score and explain correctness. Product feedback also comes from quick user loops, manual upvote/downvote signals, Slack-thread feedback, top common questions used to create evals, and planned extra validation against dashboards.

Metrics: eval pass score and reason · generated SQL versus expected SQL · generated results versus expected results · AST-normalized SQL equivalence · result-set semantic match with tolerance · top common questions represented in evals · manual upvote/downvote feedback · chain-of-thought debugging signal · dashboard-number validation for future confidence

Outcomes

• Kepler answered data questions through Slack, web UI, Cursor/MCP, and MCP platforms.
• It handled table discovery, schema checks, SQL generation, iterative querying, chart generation, anomaly debugging, follow-ups, workflows, and raw-result linking.
• The agent used table metadata, query history, lineage, code-generated descriptions, company context, memory, live warehouse calls, and evals to improve answer quality.
• Internal users described Kepler as the most useful bot, strong at writing SQL, good at sanity-checking assumptions, and close to AGI in perceived usefulness.
• The user base expanded from data scientists to GTM, finance and economics, API, Sora, ChatGPT, and the broader company.
• The team identified future work in a dedicated fine-tuned model, more validation steps, better memory visualization/conflict handling, and richer feedback analysis.

The source reports 80% of the company using 15 data-platform tools over 600PB/day and 70k datasets, a 160-line SQL example, a wrong 5M-vs-800M answer without context, 24/7 Kepler access, a 56-reply thread, 375 pickups in a follow-up demo, user/channel/global memory scopes, and expansion from data scientists to GTM, finance/econ, API, Sora, ChatGPT, and whole-company usage.