AI-Powered Code Review System at Scale

uber · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Developer productivity and code quality

Uber needed an AI code reviewer that could scale across all monorepos while keeping comments precise enough that engineers would keep using it.

Constraints:

Architecture

Uber uReview is a CI-time Phabricator reviewer with prompt-chained subtasks: comment generation, grading, semantic deduplication, category classification, developer feedback, and recurring benchmark evaluation tune usefulness at monorepo scale.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  phabricator-ci-integration["Phabricator CI integration"]
  prompt-construction["Prompt construction"]
  comment-generator["Comment generator"]
  quality-grader["Quality grader"]
  semantic-deduplication["Semantic deduplication"]
  category-classifier["Category classifier"]
  developer-feedback-stream["Developer feedback stream"]
  curated-benchmark["Curated benchmark"]
  posted-review-comments["Posted review comments"]
  phabricator-ci-integration --> prompt-construction
  prompt-construction --> comment-generator
  comment-generator --> quality-grader
  quality-grader --> semantic-deduplication
  semantic-deduplication --> category-classifier
  category-classifier --> posted-review-comments
  posted-review-comments --> developer-feedback-stream
  developer-feedback-stream --> curated-benchmark
  developer-feedback-stream --> category-classifier
  curated-benchmark --> prompt-construction
  curated-benchmark --> quality-grader
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Phabricator CI integration Runs uReview across monorepo commits as part of CI. Prompt construction
Prompt construction Builds structured prompts for code-review subtasks. Comment generator
Comment generator Uses primary review models to produce candidate code comments. Quality grader
Quality grader Validates candidate comments with a separate model configuration. Semantic deduplication
Semantic deduplication Removes near-duplicate findings before posting. Category classifier
Category classifier Tags comments and suppresses historically low-value categories. Posted review comments
Developer feedback stream Collects ratings and negative examples for filtering and benchmarks. Curated benchmark, Category classifier
Curated benchmark Evaluates prompts, thresholds, and model changes over time. Prompt construction, Quality grader
Posted review comments Appear in the developer workflow within minutes. Developer feedback stream

Data flow

uReview breaks code review into generation, filtering, validation, and deduplication; category classifiers suppress low-value comment types and feedback updates thresholds and prompts.

3 supporting quotes
“Its prompt-chaining-based architecture breaks down the code-review task into four simpler sub-tasks, and allows each sub-task—comment generation, filtering, validation, and deduplication—to evolve independently.”article, stated
“Finally, a category classifier tags each comment (for example, correctness:null-check or readability:naming) and suppresses those from categories with historically low developer value.”article, stated
“files that contain negative comments provide clear benchmark cases that future versions of uReview should avoid reproducing. Moreover, by assigning a predicted category to each comment, comment‑category filters can automatically eliminate categories”article, stated

Why this architecture

The architecture runs in CI and filters aggressively because developer workflows have low tolerance for false positives and low-value categories.

“we still want AI reviews on the code review platform (CI time) because we have less control over what the developer does locally.”article, stated
“Developers don’t like certain categories of comments from AI tools. Readability nits, minor logging tweaks, low-impact performance optimizations, and stylistic issues consistently received poor ratings.”article, stated

Models used: Anthropic Claude-4-Sonnet · OpenAI o4-mini-high · GPT-4.1

Scale

Alternatives considered (roads not taken)

Rely on local developer-side AI review only

Rejected because: Uber keeps review on the code-review platform because CI gives more control over what runs and what gets enforced.

“we still want AI reviews on the code review platform (CI time) because we have less control over what the developer does locally.”article, stated

Use AI for simple syntactic standards

Rejected because: Uber keeps linters for cheap reliable syntactic checks and uses AI for properties that linters cannot capture.

“For simple or syntactic patterns, linters are accurate, reliable, and cheap—we should continue using them. However, some properties are hard to check with linters.”article, implied

Posting every generated AI comment

Rejected because: uReview uses grading, semantic deduplication, and category filtering to suppress low-value comments.

“Finally, a category classifier tags each comment (for example, correctness:null-check or readability:naming) and suppresses those from categories with historically low developer value.”article, stated
“Developers don’t like certain categories of comments from AI tools. Readability nits, minor logging tweaks, low-impact performance optimizations, and stylistic issues consistently received poor ratings.”article, stated

Architecture-level review from code diff alone

Rejected because: Uber notes uReview lacks artifacts like past PRs, feature flags, schemas, and technical docs, so architecture-level review remains limited.

“uReview today only has access to the code, and not to other artifacts like past PRs, feature flag configurations, database schemas, technical documentation, and so on”article, stated

Patterns

prompt-chaining (agent-architecture)

uReview breaks code review into chained stages rather than depending on one monolithic review prompt.

“Its prompt-chaining-based architecture breaks down the code-review task into four simpler sub-tasks, and allows each sub-task—comment generation, filtering, validation, and deduplication—to evolve independently.”article, stated

structured-output-contract (quality-control)

The review pipeline tags comments with categories and assistant metadata, enabling structured filtering and later analysis.

“Finally, a category classifier tags each comment (for example, correctness:null-check or readability:naming) and suppresses those from categories with historically low developer value.”article, stated

data-quality-loop (feedback-loop)

Negative feedback and addressed-comment checks become benchmark cases and filter updates for future uReview versions.

“files that contain negative comments provide clear benchmark cases that future versions of uReview should avoid reproducing. Moreover, by assigning a predicted category to each comment, comment‑category filters can automatically eliminate categories”article, stated

model-routing (model-operations)

Uber pairs different models for generation and grading based on measured F1 for the specific review task.

“The most effective configuration paired Anthropic® Claude-4-Sonnet as the primary comment generator with OpenAI® o4-mini-high as the review grader.”article, stated

Tradeoffs

Where to run AI code review.

Chose CI-time comments on the code review platform over depending only on IDE/editor integrations.

Rationale: CI gives Uber more consistent enforcement than relying on local developer behavior.

“we still want AI reviews on the code review platform (CI time) because we have less control over what the developer does locally.”article, stated

How to handle simple coding standards.

Chose LLMs for semantic cases and linters for syntactic patterns over replacing static analysis wholesale.

Rationale: The case frames GenAI as complementary to cheap deterministic tools.

“For simple or syntactic patterns, linters are accurate, reliable, and cheap—we should continue using them. However, some properties are hard to check with linters.”article, stated

Gotchas (what bit them)

Code-level review does not automatically solve architecture-level review.

Root cause: uReview currently lacks context such as past PRs, feature flags, schemas, and technical documentation.

Resolution: The team treats source-code-only review as the current boundary and points to richer MCP-style context as future work.

“uReview today only has access to the code, and not to other artifacts like past PRs, feature flag configurations, database schemas, technical documentation, and so on”article, stated

AI comments in low-value categories create developer fatigue.

Root cause: Readability nits, logging tweaks, stylistic issues, and low-impact performance suggestions received poor ratings.

Resolution: Uber suppresses historically low-value categories and focuses on correctness, error handling, and best-practice violations.

“Developers don’t like certain categories of comments from AI tools. Readability nits, minor logging tweaks, low-impact performance optimizations, and stylistic issues consistently received poor ratings.”article, stated

Evaluation

Uber evaluates addressed comments in production, uses a curated benchmark for precision/recall/F1, and periodically retests model configurations.

Metrics: usefulness rate · addressed-comment rate · precision · recall · F1 · median latency

“Feedback from both methods informs adjustments to confidence thresholds, prompts, and filtering logic. We aim to maintain a usefulness rate above 75% as the system expands”article, stated

Outcomes

The article reports CI deployment across Go, Java, Android, iOS, TypeScript, and Python monorepos with median review time of 4 minutes.

“uReview is now deployed across all six of Uber’s monorepos (Go, Java, Android, iOS, Typescript, and Python), and reviews every commit as part of our CI process within a median of 4 minutes.”article, stated
“This translates to approximately 1,500 hours saved weekly, equivalent to nearly 39 developer years annually.”article, stated