AI-Powered Data Copilot for Autonomous Analysis in IDEs

blablacar · Tech · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.0 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Engineering-owned data analysis

BlaBlaCar needed to reduce the friction between software engineers and data analysts so engineers could answer team data questions safely without overwhelming analysts with repetitive ad hoc requests.

Constraints:

Engineers know product/domain code but may fear SQL, DBT, Airflow, and data-warehouse tooling
Data analysts need to avoid becoming a query factory
Analyses must be reproducible, reviewed, and grounded in business logic

Architecture

BlaBlaCar Data Copilot is a zero-infrastructure RAG system embedded in VS Code. A lightweight Python script securely bridges BigQuery and the IDE by exporting schema definitions, golden queries, table samples, DDL, KPI SQL, and generated documentation into Markdown/SQL/JSON files inside the repository. VS Code indexes those files, and GitHub Copilot uses #codebase semantic search and #textSearch literal search to inject relevant context. The assistant generates SQL/Python analyses, unit-test assertions, and Data Health Cards; engineers commit the artifacts and data analysts review them as pull requests. Merged PRs become repository memory for future analyses.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  bigquery-warehouse[BigQuery warehouse] --> python-context-bridge[Python context bridge]
  python-context-bridge --> repository-context-files[Repository context files]
  curated-golden-queries[Curated golden queries] --> repository-context-files
  repository-context-files --> vs-code-ide[VS Code IDE]
  vs-code-ide --> github-copilot[GitHub Copilot]
  repository-context-files --> codebase-search[#codebase search]
  repository-context-files --> text-search[#textSearch]
  codebase-search --> github-copilot
  text-search --> github-copilot
  github-copilot --> analysis-script[Analysis script]
  analysis-script --> unit-tests[Unit tests]
  analysis-script --> data-health-card[Data Health Card]
  analysis-script --> pull-request-review[Pull request review]
  data-health-card --> pull-request-review
  pull-request-review --> repository-memory[Repository memory]
  repository-memory --> github-copilot

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`vs-code-ide`	Engineer workspace where the copilot runs, context is indexed, and analyses are edited/reviewed like code.	`github-copilot`, `repository-context-files`
`github-copilot`	LLM assistant using VS Code tools to search repository context and generate SQL/Python analysis code.	`vs-code-ide`, `codebase-search`, `text-search`, `analysis-script`
`bigquery-warehouse`	Authorized data warehouse source for schemas, table samples, and queryable data.	`python-context-bridge`
`python-context-bridge`	Lightweight script that tunnels securely into BigQuery and exports context into repository files.	`bigquery-warehouse`, `repository-context-files`
`repository-context-files`	Markdown, SQL, and JSON files containing schemas, golden queries, table samples, DDL, KPI SQL, and generated documentation.	`python-context-bridge`, `vs-code-ide`, `codebase-search`, `text-search`
`curated-golden-queries`	Production DBT/reporting examples that encode business definitions such as churn or search intent.	`repository-context-files`
`codebase-search`	#codebase semantic search over workspace context for intent retrieval.	`github-copilot`, `repository-context-files`
`text-search`	#textSearch literal matching against generated documentation and query files.	`github-copilot`, `repository-context-files`
`analysis-script`	Generated SQL/Python artifact containing the analysis logic and parameters users can inspect or modify.	`github-copilot`, `unit-tests`, `pull-request-review`
`unit-tests`	Auto-generated assertions that make analyses reviewable and less ephemeral.	`analysis-script`
`data-health-card`	Heuristic soft-warning layer that catches analytically risky joins, deprecated fields, or data-quality issues.	`analysis-script`, `pull-request-review`
`pull-request-review`	Version-control gate where engineers commit scripts and data analysts review code, tests, and analysis logic.	`analysis-script`, `data-health-card`, `repository-memory`
`repository-memory`	Merged analysis scripts indexed for reuse and refresh by future copilot queries.	`pull-request-review`, `github-copilot`

Data flow

The Python bridge pulls authorized BigQuery context, curated production DBT/reporting queries, schemas, and table samples into repository text files. VS Code indexes those files automatically. When an engineer asks a data question, GitHub Copilot searches the workspace with #codebase and #textSearch, injects relevant examples/DDL/KPI SQL into chat context, and generates the raw SQL and Python needed for the analysis. The tool also generates unit tests and a Data Health Card. The engineer commits the script, the data analyst reviews the pull request, and merged analyses are later indexed as long-term memory for similar questions.

5 supporting quotes

“A lightweight Python script bridges BigQuery and the IDE. It exports key context, schema definitions, Golden queries, and table samples, into standard text files (Markdown, SQL, JSON) directly within the project’s repository.”article, stated

“Because these are simply files in the workspace, they are automatically indexed by VS Code, making them instantaneously available to GitHub Copilot.”article, stated

“It triggers #codebase to perform a semantic search across the workspace, identifying the intent behind the query.”article, stated

“With Data Copilot, every analysis is generated as a Python script with auto-generated Unit Tests (assertions).”article, stated

“The Copilot indexes every merged Pull Request, meaning the repository effectively becomes its long-term memory.”article, stated

Why this architecture

BlaBlaCar designed for engineers, not generic business users, so the assistant lives in the IDE and emits code artifacts rather than opaque BI outputs. The zero-infrastructure RAG approach deliberately avoids vector databases and MCP servers because VS Code and GitHub Copilot already provide indexing and retrieval over workspace files. Pull requests, unit tests, Data Health Cards, and repository memory make analysis transparent, reviewable, reusable, and aligned with software-engineering practice.

4 supporting quotes

“We built a tool designed explicitly for Engineers, living where they live: In the IDE.”article, stated

“We bypass complex infrastructure like vector databases or separate ‘Model Context Protocol’ (MCP) servers. Instead, we leverage the native capabilities of the engineer’s existing environment.”article, stated

“Unlike standard BI tools that hide logic behind drag-and-drop interfaces, Our Data Copilot treats analysis as artifacts generated by a unique composition of Code and LLM.”article, stated

“This transforms the DA from a Query Factory into a Reviewer and Guide.”article, stated

Models used: GitHub Copilot models

Scale

The tool targets an organizational bottleneck where analysts were buried under repetitive quick questions.
“But they are often buried under a mountain of “quick questions”.”article, stated
The repository-memory loop means each merged analysis can reduce future work by seeding similar questions.
“If an engineer asks a question similar to one answered last quarter, the Copilot can surface the original script as a starting point.”article, stated
The reported time-to-answer improvement is from Jira backlog scale to IDE-session scale for suitable questions.
“Questions that used to sit in a Jira backlog for 3 weeks are now answered in 10 minutes inside VS Code.”article, stated
An open-source version exists for BigQuery sample datasets or custom warehouses, suggesting the approach is intended to be portable beyond BlaBlaCar.
“This lightweight version allows you to connect to BigQuery sample datasets or to your own Data Warehouse and start asking questions.”article, stated

Alternatives considered (roads not taken)

A business-user text-to-SQL chatbot

Rejected because: BlaBlaCar wanted engineers to analyze data where they already write and review code, not replace analysts with a separate chatbot.

“Most Text-to-SQL tools on the market today are designed for business users, replacing the analyst with a chat bot. We took a different approach.”article, stated

Vector databases or separate MCP servers for RAG

Rejected because: The team used repository files plus VS Code indexing/Copilot retrieval to avoid complex new infrastructure.

“We bypass complex infrastructure like vector databases or separate ‘Model Context Protocol’ (MCP) servers.”article, stated

Ephemeral BI charts or Slack screenshots

Rejected because: Those artifacts are hard to reproduce and review; code plus tests and PRs preserve logic and reviewer context.

“In the old world, an analysis was a screenshot of a chart pasted into Slack. It was ephemeral, unverified, and hard to reproduce.”article, stated

Patterns

`rag-retrieval` (ide-context-retrieval)

The copilot retrieves schema, KPI, DDL, and golden-query files from the workspace as context for analysis questions.

“It surfaces files like tables/kpis_mau/usage.sql or tables/carpool_members/ddl.sql and injects their content directly into the chat context.”article, stated

`context-compaction` (zero-infrastructure-context)

BlaBlaCar avoids vector databases and MCP servers by exporting only key context into simple repo files indexed by the IDE.

“We bypass complex infrastructure like vector databases or separate ‘Model Context Protocol’ (MCP) servers.”article, stated

`structured-output-contract` (analysis-as-code)

The assistant emits Python scripts and unit tests instead of ephemeral charts, turning analysis into reviewable code artifacts.

“With Data Copilot, every analysis is generated as a Python script with auto-generated Unit Tests (assertions).”article, stated

`guardrails-io-filtering` (data-health)

Data Health Cards run heuristic checks and soft warnings so engineers can move quickly while learning to catch bad data patterns.

“Our Copilot solves this by generating a Data Health Card alongside every analysis. It runs heuristic checks providing a Soft Warning system.”article, stated

`human-in-the-loop-gate` (analysis-review)

Data analysts review pull requests containing generated analysis code and tests rather than answering every question directly.

“The Data Analyst reviews the Pull Request.”article, stated

`agent-memory` (repository-memory)

Merged pull requests become reusable long-term memory for future similar analyses.

“The Copilot indexes every merged Pull Request, meaning the repository effectively becomes its long-term memory.”article, stated

Tradeoffs

Where to build the copilot.

Chose inside the engineer's IDE over a business-user chatbot or separate BI interface.

Rationale: The goal was to help engineers run rigorous analyses where they already write and review code.

“We built a tool designed explicitly for Engineers, living where they live: In the IDE.”article, stated

How much AI infrastructure to build.

Chose workspace files indexed by VS Code/GitHub Copilot over custom vector databases or MCP servers.

Rationale: The lightweight approach used existing IDE capabilities and avoided a complex new platform.

“Because these are simply files in the workspace, they are automatically indexed by VS Code”article, stated

How strict data-quality guardrails should be for engineer-led analysis.

Chose Data Health Cards with soft warnings over hard blocking every potentially risky analysis pattern.

Rationale: Soft warnings let engineers move quickly while learning common data pitfalls; analysts still review complex outputs in pull requests.

“It runs heuristic checks providing a Soft Warning system. This allows engineers to move fast while passively learning to spot bad data patterns without being blocked.”article, stated

How to make generated analysis auditable.

Chose SQL/Python code plus unit tests committed through pull requests over static charts or hidden BI logic.

Rationale: Code artifacts make the logic inspectable, modifiable, testable, and reviewable by data analysts.

“The DA doesn’t just see a chart; they see the code and the passing tests.”article, stated

Gotchas (what bit them)

Engineers were blocked by unfamiliar data tooling, not lack of analytical skill.

Root cause: SQL dialects, DBT syntax, Airflow operators, and organizational silos made the warehouse feel unsafe.

Resolution: BlaBlaCar put the copilot in the IDE and gave engineers contextual business logic plus guardrails.

“The barrier isn’t a lack of skill; it is the friction of unfamiliar tooling, the fear of “doing it wrong” without a safety net”article, stated

Syntactically valid queries can still be analytically wrong.

Root cause: SQL may run while violating business definitions, data health assumptions, or model semantics.

Resolution: Data Health Cards provide heuristic checks and soft warnings next to each analysis.

“In data, a query can be syntactically perfect but analytically disastrous.”article, stated

Slack-shared screenshots made analysis ephemeral and unreproducible.

Root cause: Charts pasted into chat lacked code, tests, review, and repository memory.

Resolution: The copilot generates scripts/tests committed through pull requests and indexed for reuse.

“In the old world, an analysis was a screenshot of a chart pasted into Slack. It was ephemeral, unverified, and hard to reproduce.”article, stated

Evaluation

BlaBlaCar evaluates by time-to-answer, analyst workload shift, pull-request review, generated unit tests/assertions, Data Health Cards, and reuse of repository memory.

Metrics: question turnaround time · pull-request review · unit-test assertions · data-health warnings · analyst support-queue reduction · analysis reuse

“Questions that used to sit in a Jira backlog for 3 weeks are now answered in 10 minutes inside VS Code.”article, stated

“They spend less time writing SELECT * and more time guiding engineers toward rigorous analytical standards.”article, implied

Outcomes

Questions that waited three weeks in Jira can be answered in about 10 minutes inside VS Code.
Engineers can validate feature ideas before building them.
Analysts shift toward review, mentoring, modeling, and platform improvement.
Analyses become code artifacts with unit tests and repository memory.

The tool targets BlaBlaCar's software-engineering and data-analyst workflow, bridging BigQuery and IDE-based development practices.

“Freed from the support queue, our DAs are focusing on deep-dive modeling and improving the core data platform.”article, stated

“Composable Knowledge: A complex segmentation logic built by a Senior DA for one project becomes a reusable module for an SWE’s future query.”article, stated