AI-Powered Code Review Platform at Scale

uber · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.0 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Developer productivity and code quality

Uber needed code-review assistance that could keep up with tens of thousands of weekly diffs without eroding developer trust through noisy or false-positive comments.

Constraints:

Six large monorepos and Phabricator-based review workflows
False positives quickly reduce developer trust
Feedback must arrive quickly enough to help before human review

Architecture

uReview is a CI-time, prompt-chained code-review platform integrated with Uber’s code-review workflow. It filters eligible diffs, builds structured prompts with surrounding code context, routes files through a pluggable set of specialized assistants, grades generated comments with a secondary model, merges duplicates, classifies/suppresses low-value categories, posts validated inline comments, and streams comment metadata plus developer feedback into Kafka/Hive. Automated addressed-comment checks and curated benchmarks feed threshold, prompt, category, and model updates.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  code-review-platform[Code review platform] --> ingestion-and-preprocessing[Ingestion and preprocessing]
  ingestion-and-preprocessing --> structured-prompt-builder[Structured prompt builder]
  structured-prompt-builder --> pluggable-assistant-framework[Pluggable assistant framework]
  pluggable-assistant-framework --> standard-assistant[Standard Assistant]
  pluggable-assistant-framework --> best-practices-assistant[Best Practices Assistant]
  best-practices-assistant --> style-rule-registry[Style rule registry]
  pluggable-assistant-framework --> appsec-assistant[AppSec Assistant]
  pluggable-assistant-framework --> quality-grader[Quality grader]
  quality-grader --> semantic-similarity-filter[Semantic similarity filter]
  semantic-similarity-filter --> category-classifier[Category classifier]
  category-classifier --> comment-delivery[Comment delivery]
  comment-delivery --> code-review-platform
  comment-delivery --> feedback-data-platform[Feedback data platform]
  feedback-data-platform --> evaluation-loop[Evaluation loop]
  evaluation-loop --> pluggable-assistant-framework

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`code-review-platform`	Uber’s review platform where developers submit diffs and where uReview posts inline comments.	`ingestion-and-preprocessing`, `comment-delivery`
`ingestion-and-preprocessing`	Filters configuration/generated/experimental targets and selects eligible files for review.	`code-review-platform`, `structured-prompt-builder`
`structured-prompt-builder`	Constructs prompts with nearby functions, class definitions, imports, and other code context.	`ingestion-and-preprocessing`, `pluggable-assistant-framework`
`pluggable-assistant-framework`	Runs specialized assistants that can be developed, evaluated, and tuned independently.	`structured-prompt-builder`, `standard-assistant`, `best-practices-assistant`, `appsec-assistant`, `quality-grader`
`standard-assistant`	Detects bugs, incorrect exception handling, and logic flaws.	`pluggable-assistant-framework`
`best-practices-assistant`	Enforces Uber-specific coding conventions via a shared style-rule registry.	`pluggable-assistant-framework`, `style-rule-registry`
`appsec-assistant`	Targets application-level security vulnerabilities.	`pluggable-assistant-framework`
`style-rule-registry`	Registry of internal style and best-practice rules referenced by the Best Practices Assistant.	`best-practices-assistant`
`quality-grader`	Secondary prompt/model that scores each generated comment and supports pruning thresholds.	`pluggable-assistant-framework`, `semantic-similarity-filter`
`semantic-similarity-filter`	Merges overlapping suggestions so developers see one concise comment instead of duplicates.	`quality-grader`, `category-classifier`
`category-classifier`	Tags comment categories and suppresses categories with historically low developer value.	`semantic-similarity-filter`, `comment-delivery`, `feedback-data-platform`
`comment-delivery`	Posts validated comments inline and collects Useful/Not Useful ratings and optional notes.	`code-review-platform`, `feedback-data-platform`
`feedback-data-platform`	Kafka/Hive data path for comment metadata, developer feedback, dashboards, experiments, and benchmark case generation.	`comment-delivery`, `evaluation-loop`
`evaluation-loop`	Automated five-rerun addressed-comment checks plus curated precision/recall/F1 benchmarks for model and pipeline tuning.	`feedback-data-platform`, `pluggable-assistant-framework`

Data flow

When a developer submits a Phabricator change, uReview filters out low-signal files and builds structured prompts around eligible code. Specialized assistants generate comments for bugs, best practices, or AppSec issues. A secondary prompt assigns confidence scores, semantic filtering merges overlap, and a category classifier suppresses historically low-value categories. Validated comments are posted inline on the review platform; usefulness ratings, confidence, category, assistant origin, and developer notes stream through Kafka to Hive. Later, uReview re-runs on final commits and uses curated benchmarks to tune prompts, thresholds, filters, and model selection.

4 supporting quotes

“When a developer submits a change on Uber’s code review platform, uReview first determines which files are eligible for automated review.”article, stated

“uReview uses a pluggable assistant framework, where each assistant focuses on a specific class of issues.”article, stated

“Once comments are generated, uReview runs them through a multi-layered filtering process:”article, stated

“All comments, along with their associated metadata—including assistant origin, category, confidence score, and developer feedback—are streamed to Apache Hive™ via Apache Kafka®.”article, stated

Why this architecture

The architecture optimizes for trust rather than maximum comment volume. Uber decomposes code review into independently tunable stages because false positives come from both hallucination and contextually low-value true positives, and a single prompt cannot reliably handle generation, filtering, validation, deduplication, feedback, and rollout. The system also stays inside CI/code review rather than relying only on local IDE behavior, because Uber can enforce and measure that path consistently across teams.

4 supporting quotes

“A high false‑positive rate undermines engineers’ perception of the tool’s accuracy and usefulness—when they encounter many false-positive comments, they start to tune out and ignore them.”article, stated

“Its prompt-chaining-based architecture breaks down the code-review task into four simpler sub-tasks, and allows each sub-task—comment generation, filtering, validation, and deduplication—to evolve independently.”article, stated

“Prompt design helped, but system architecture, and post-processing were even more critical.”article, stated

“we still want AI reviews on the code review platform (CI time) because we have less control over what the developer does locally.”article, stated

Models used: Anthropic Claude-4-Sonnet · OpenAI o4-mini-high

Scale

uReview analyzes over 90% of roughly 65,000 weekly diffs, so even modest false-positive rates would create substantial developer noise.
“uReview today analyzes over 90% of the weekly ~65,000 diffs (equivalent of pull requests) landed at Uber.”article, stated
The system is deployed across six monorepos and supports Go, Java, Android, iOS, TypeScript, and Python.
“uReview is now deployed across all six of Uber’s monorepos (Go, Java, Android, iOS, Typescript, and Python), and reviews every commit as part of our CI process within a median of 4 minutes.”article, stated
The feedback/evaluation path runs at production scale, with automated addressed-comment checks on thousands of commits daily.
“The advantage of ‌automatic feedback is that it runs on thousands of commits in production daily.”article, stated
Uber estimates a second human reviewer for comparable issues would cost about 1,500 hours weekly.
“This translates to approximately 1,500 hours saved weekly, equivalent to nearly 39 developer years annually.”article, stated

Alternatives considered (roads not taken)

Third-party AI code-review tools

Rejected because: They generally required GitHub, performed poorly on Uber code, and could not interact with Uber internal systems.

“First, most third-party AI code-review tools require code to be hosted on GitHub®.”article, stated

“our evaluation of third-party tools on Uber code showed that they suffered from three main issues: many false positives, low-value true positives, and being unable to interact with internal systems at Uber.”article, stated

Single-shot prompting for code review

Rejected because: Unfiltered outputs caused hallucinated issues, duplicate suggestions, and inconsistent quality.

“Even with high-performing models like o4-mini-high and Claude 4 Sonnet, single-shot prompting wasn’t enough. Unfiltered outputs led to hallucinated issues, duplicate suggestions, and inconsistent quality.”article, stated

IDE-only or local AI review

Rejected because: The CI/code-review platform path gives Uber more control than hoping developers use and heed local review warnings.

“They may not use the AI code-review features or may ignore its warnings. This is analogous to the concept of running build and test at CI time in addition to making build and test available for developers locally.”article, stated

Traditional linters for all best-practice enforcement

Rejected because: Linters are better for cheap syntactic checks, but semantic best-practice cases require understanding code meaning.

“For simple or syntactic patterns, linters are accurate, reliable, and cheap—we should continue using them. However, some properties are hard to check with linters.”article, stated

Patterns

`prompt-chaining` (agent-architecture)

Uber decomposes AI code review into generation, filtering, validation, and deduplication stages so each part can be tuned independently.

“uReview is a modular, multi-stage GenAI system designed to automate and enhance code reviews across Uber’s engineering platforms. Its prompt-chaining-based architecture breaks down the code-review task into four simpler sub-tasks”article, stated

`confidence-thresholding` (quality-control)

Generated comments are graded by a secondary prompt and pruned with thresholds tuned by assistant, language, and comment category.

“A secondary prompt evaluates each comment’s quality and assigns a confidence score. The prompt is customized for each assistant type, and confidence thresholds for pruning are set at a fine-grained level”article, stated

`guardrails-io-filtering` (deduplication)

The pipeline applies a semantic similarity filter to collapse overlapping review comments before showing them to developers.

“2. Next, a semantic similarity filter merges overlapping suggestions. 3. Finally, a category classifier tags each comment”article, stated

`data-quality-loop` (feedback-loop)

Developer ratings, category metadata, confidence scores, and addressed-comment signals feed future filtering and benchmark cases.

`eval-driven-development` (evaluation)

Uber evaluates model and grader combinations with automated addressed-comment checks and human-labeled precision/recall/F1 benchmarks.

“Manually, a curated benchmark of commits with known issues is used to evaluate comment precision, recall, and F1 scores”article, stated

Tradeoffs

How aggressively to surface AI review comments.

Chose fewer, higher-precision comments over maximizing comment volume or recall.

Rationale: Uber found that quality mattered more than quantity for maintaining developer engagement.

“Early in development, we learned that comment quality matters far more than quantity. Developers quickly lose confidence in a tool that generates low-quality or irrelevant suggestions.”article, stated

Whether to buy an AI code-review product.

Chose an in-house reviewer specialized to Uber over third-party AI code-review tools.

Rationale: Third-party tools were tightly coupled to GitHub, showed false positives, and could not use Uber internal systems.

Where to run AI review in the developer workflow.

Chose CI-time review on the code-review platform over relying only on IDE or local review integrations.

Rationale: Uber wanted a controlled, consistently enforced path analogous to CI builds/tests, because local tool usage is optional and warnings may be ignored.

“Even though some IDEs (or extensions) offer code reviews, we still want AI reviews on the code review platform (CI time) because we have less control over what the developer does locally.”article, stated

When to use GenAI instead of static analysis for best practices.

Chose LLMs for semantic best-practice cases that linters cannot encode well over replacing all linters with GenAI checks.

Rationale: The team explicitly keeps linters for cheap syntactic rules and uses LLMs where semantic understanding is needed.

“For simple or syntactic patterns, linters are accurate, reliable, and cheap—we should continue using them. However, some properties are hard to check with linters.”article, stated

How many stochastic reruns to use for addressed-comment evaluation.

Chose five reruns of uReview on the final commit over a single rerun or no automated addressed-comment check.

Rationale: Five reruns were chosen as the minimum that reduced missed detections while keeping evaluation cost and latency low.

“we invoke it five times—the minimal count that virtually eliminates missed detections while keeping cost and latency low.”article, stated

Gotchas (what bit them)

False positives can make engineers ignore the AI reviewer.

Root cause: LLMs can hallucinate incorrect comments, and even valid comments may be irrelevant in a specific code context.

Resolution: uReview applies quality scoring, category pruning, deduplication, and feedback-informed thresholds before posting comments.

“The main challenge of AI code review is false positives from two sources: LLM hallucinations that generate incorrect comments and issues that are generally valid but not important in that specific scenario”article, stated

Single-shot prompting was insufficient for production-grade code review.

Root cause: Unfiltered model outputs produced hallucinated issues, duplicate suggestions, and uneven quality.

Resolution: Uber introduced chained generation, grading, filtering, and consolidation stages around the model.

Evaluation

Uber combines production addressed-comment checks with curated human-labeled benchmarks and model-combination F1 comparisons.

Metrics: usefulness rate · comment addressed rate · precision · recall · F1 · median review latency

“It automatically evaluates if a given posted comment has been addressed by re-running uReview five times on the final commit. Because LLM is stochastic, a single rerun might skip a lingering issue”article, stated

“The most effective configuration paired Anthropic® Claude-4-Sonnet as the primary comment generator with OpenAI® o4-mini-high as the review grader. This combination achieved the highest F1 score”article, stated

Outcomes

uReview analyzes more than 90% of weekly diffs.
Engineers mark 75% of comments useful and address over 65% of posted comments.
Uber estimates approximately 1,500 engineering hours saved weekly.

The system is deployed across all six Uber monorepos and reviews commits in CI with median latency around 4 minutes.

“uReview today analyzes over 90% of the weekly ~65,000 diffs (equivalent of pull requests) landed at Uber. Engineers who interact with the tool mark 75% of its comments as useful”article, stated

“Each week, uReview processes over 10,000 commits, excluding configuration files. Internal benchmarks indicate that having a second human reviewer look for the kinds of issues identified by uReview would require 10 minutes per commit.”article, stated