AI-Powered Developer Tools for Code Quality and Test Generation

uber · Tech · year n/a · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: AI developer tools for code quality and tests

Uber needed AI-powered developer tools that improve code quality and test coverage for thousands of developers working on a codebase with hundreds of millions of lines.

Constraints:

The tools needed to serve about 5,000 developers.
Uber's codebase contained hundreds of millions of lines of code.
The products had to integrate into existing IDE and build workflows.
Generated tests needed to pass execution and mutation testing, not just compile.
Reusable agent primitives were needed across multiple developer products.

Architecture

Uber built Validator and AutoCover as LangGraph-based developer tools. Validator combines LLM best-practice analysis with deterministic static linters and IDE fixes. AutoCover orchestrates scaffolding, test generation, execution, coverage checks, and Validator reuse to stream validated tests into the IDE.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  developer_ide["Developer IDE"] --> validator_agent["Validator agent"]
  llm_best_practice_sub_agent["LLM best-practice sub-agent"] --> validator_agent
  static_linter_component["Static linter component"] --> validator_agent
  developer_ide --> autocover_workflow["AutoCover workflow"]
  autocover_workflow --> scaffolder["Scaffolder"]
  autocover_workflow --> generator["Generator"]
  generator --> executor["Executor"]
  executor --> autocover_workflow
  autocover_workflow --> validator_agent

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Developer IDE`	Surfaces diagnostics, fixes, and generated tests to engineers.	`Validator agent`, `AutoCover workflow`
`Validator agent`	Flags best-practice and security issues and validates generated tests.	`Developer IDE`, `LLM best-practice sub-agent`, `Static linter component`, `AutoCover workflow`
`LLM best-practice sub-agent`	Evaluates code against curated best practices.	`Validator agent`
`Static linter component`	Provides deterministic code findings.	`Validator agent`
`AutoCover workflow`	Generates, runs, filters, and validates tests.	`Developer IDE`, `Scaffolder`, `Generator`, `Executor`, `Validator agent`
`Scaffolder`	Prepares test targets and identifies business cases worth testing.	`AutoCover workflow`
`Generator`	Creates new or extended test cases.	`AutoCover workflow`, `Executor`
`Executor`	Runs builds, tests, and coverage checks.	`Generator`, `AutoCover workflow`

Data flow

Validator runs when a developer opens a file, combining LLM and linter findings into IDE diagnostics with pre-computed fixes or agent-assistant handoff. AutoCover starts from a source file, scaffolds a test target, creates a test file, checks baseline coverage, analyzes business context, generates tests, executes them, removes failures, merges redundant tests, and reuses Validator.

3 supporting quotes

“Validator is an IDE-integrated experience that automatically flags best practice violations and security issues in code.”summary, stated

“AutoCover is a more complex tool designed to automatically generate high-quality, passing tests that raise code coverage.”summary, stated

“they run static linters and pass those findings through the rest of the graph.”summary, stated

Why this architecture

Uber used graph-based composition because developer tools share reusable pieces: deterministic linters, LLM review, scaffolding, generation, execution, and validation can be recombined across products.

“The solution involved creating domain-expert agents for tasks like code validation, automated test generation, and code review, composed together using LangGraph's graph-based architecture.”summary, stated

“Intentional Tech Transfer: Perhaps the most interesting aspect of their strategy is the deliberate effort to identify reusable components from individual products and spin them out as shared primitives.”summary, stated

Models used: LangGraph agents · LangChain · LLM-powered sub-agents

Scale

Uber served approximately 5,000 developers.
“serving approximately 5,000 developers working on a codebase with hundreds of millions of lines of code.”summary, stated
Uber processes 33 million trips daily across 15,000 cities.
“The company processes 33 million trips daily across 15,000 cities”summary, stated
AutoCover improved developer-platform coverage by 10%, equivalent to 21,000 developer hours saved.
“10% coverage improvement for the developer platform (equivalent to 21,000 dev hours saved)”summary, stated
AutoCover generated over 5,000 tests weekly.
“5,000+ tests generated weekly”summary, stated
AutoCover achieved 2-3x more coverage in half the time versus alternatives.
“achieving 2-3x more coverage in half the time compared to industry alternatives.”summary, stated

Alternatives considered (roads not taken)

Use LLM-only code analysis

Rejected because: Uber combined LLM sub-agents with deterministic static linters.

“This hybrid approach of combining LLM-based analysis with deterministic tooling is a pattern that recurs throughout their architecture.”summary, stated

Generate syntactically correct tests only

Rejected because: AutoCover validates tests through execution and mutation testing.

“The emphasis is on generating tests that are validated through actual execution and mutation testing, not just syntactically correct code.”summary, stated

Build isolated one-off AI tools

Rejected because: The team extracted shared primitives such as Lang Effect.

“creation of "Lang Effect," their internal opinionated framework that wraps LangGraph and LangChain”summary, stated

Patterns

`multi-agent-orchestration` (developer tooling)

Validator and AutoCover compose specialized agents for analysis, scaffolding, generation, execution, and validation.

“The underlying graph architecture is particularly sophisticated. It includes:”summary, stated

`tool-use-orchestration` (build/test automation)

AutoCover runs build, test, coverage, and validation tools as part of the graph.

“• Executor: Runs builds, tests, and coverage checks”summary, stated

`eval-driven-development` (generated tests)

Generated tests are executed and mutation-tested before being accepted.

“The emphasis is on generating tests that are validated through actual execution and mutation testing, not just syntactically correct code. The goal is saving developer time so engineers can focus on implementing business features rather than writing boilerplate tests.”summary, stated

`human-in-the-loop-gate` (developer review)

Developers choose whether to apply pre-computed fixes or send requests to an IDE assistant.

“developers can either apply a pre-computed fix that was prepared in the background, or they can send the fix request to their IDE's agentic assistant.”summary, stated

Tradeoffs

Integrate into existing developer workflows.

Chose IDE diagnostics and right-click AutoCover invocation over A separate AI coding portal.

Rationale: Meeting developers where they work reduces adoption friction.

“The user experience is well-considered: developers can either apply a pre-computed fix that was prepared in the background, or they can send the fix request to their IDE's agentic assistant.”summary, stated

Extract reusable primitives from products.

Chose Lang Effect and shared agents over Duplicate agent code in each product.

Rationale: Reusable primitives accelerate new AI developer tools.

“Cross-cutting Primitives: Building foundational AI technologies and abstractions that can be reused across multiple solutions. This includes developing the right frameworks and tooling that accelerate the development of new AI-powered features.”summary, stated

Gotchas (what bit them)

Generated tests need execution, not just syntax.

Root cause: LLMs can write plausible tests that fail, duplicate coverage, or miss business cases.

Resolution: AutoCover runs builds, tests, coverage, mutation testing, and validation loops.

“failing tests get removed, tests may be merged or marked redundant, and eventually benchmark and concurrency tests appear.”summary, stated

Pure LLM code review misses deterministic signals.

Root cause: Static analysis already catches known classes of issues reliably.

Resolution: Combine LLM reasoning with static linter findings in the same graph.

“they also incorporate deterministic components—specifically, they run static linters and pass those findings through the rest of the graph.”summary, stated

Evaluation

Uber evaluated developer tools by daily fix interactions, coverage improvement, developer hours saved, weekly tests generated, and relative coverage/time against alternatives.

Metrics: Thousands of daily fix interactions. · 10% developer-platform coverage improvement. · 21,000 developer hours saved. · 5,000+ tests generated weekly. · 2-3x more coverage in half the time.

“The impact metrics are impressive: thousands of fix interactions daily from engineers resolving code issues before they become larger problems.”summary, stated

“Results include thousands of daily fix interactions, 10% coverage improvement for the developer platform (equivalent to 21,000 dev hours saved), and 5,000+ tests generated weekly, with the test generation tool achieving 2-3x more coverage in half the time compared to industry alternatives.”summary, stated

Outcomes

Uber deployed Validator for code quality and security diagnostics.
AutoCover generated validated tests and improved coverage.
LangGraph-based primitives were reused across developer tools.

The tools serve roughly 5,000 developers on a codebase with hundreds of millions of lines.

“built a suite of AI-powered developer tools using LangGraph to improve code quality and developer productivity”summary, stated