Automated Performance Optimization with GenAI-Powered Code Analysis

uber · Tech · 2025 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Software performance engineering

Uber needed to find and fix Go performance antipatterns across high-cost production services without relying on scarce compiler/runtime experts and multi-day manual profiling cycles.

Constraints:

• the top Go services represented multi-million-dollar monthly compute spend
• manual profiling and optimization required deep Go/runtime expertise
• LLM detections needed high precision before developers would trust them
• hotpath code can be large enough to require high input token limits
• recommendations must fit CI/CD and developer workflows rather than one-off audits

Architecture

Uber built PerfInsights as a two-stage system. First, a fleet-wide profiler identifies the top 30 expensive Go functions during peak traffic, adds memory profiling when runtime.mallocgc exceeds 15% CPU time, and statically filters out OSS dependencies and runtime internals so only service-owned hotpaths remain. Second, a GenAI detector receives full source code plus a curated Go antipattern catalog and proposes optimizations. Few-shot prompts, Go-expert role framing, confidence scoring, LLM juries, and LLMCheck rule validators reduce hallucinations and false positives. Validated suggestions flow to downstream code transformation or manual review, and integration with Optix/CI turns performance tuning into a continuous workflow.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  fleet-wide-profiler[Fleet-wide profiler] --> static-hotpath-filter[Static hotpath filter]
  static-hotpath-filter --> genai-antipattern-detector[GenAI antipattern detector]
  genai-antipattern-detector --> llm-jury[LLM jury]
  llm-jury --> llmcheck-validators[LLMCheck validators]
  llmcheck-validators --> optix-code-optimization[Optix code optimization]
  optix-code-optimization --> developer-review-workflow[Developer review workflow]
  developer-review-workflow --> llmcheck-dashboards[LLMCheck dashboards]

Components

Data flow

Production CPU and memory profiles feed a static filtering step that selects expensive service-owned functions. PerfInsights sends each candidate function and an antipattern catalog to an LLM detector. The output is checked by LLM juries and LLMCheck validators, scored for confidence, and sent to code transformation or developer review. Metrics from LLMCheck dashboards track false positives, regressions, antipattern frequency, and code-health trends.

Why this architecture

PerfInsights deliberately combines deterministic profiling/static filtering with LLM analysis because raw LLM scans over a monorepo would be noisy and costly. The profiler finds code paths worth optimizing, the static filter constrains the context to service-owned functions, and the LLM brings pattern-recognition ability to code-level antipattern detection. LLM juries and LLMCheck are separate trust layers because early single-shot attempts hallucinated and generated unreliable outputs.

Models used: large language models · LLM juries · GenAI model

Scale

• In March 2024, Uber's top 10 Go services accounted for more than multi-million dollars in compute spend.
  “In March 2024 alone, the top 10 Go services accounted for more than multi-million dollars in compute spend alone”article, stated
• PerfInsights selects the top 30 most expensive functions per service.
  “For each service, it identifies the top 30 most expensive functions based on flat CPU usage. This is based on the observation that the top 30 most expensive functions account for the majority of CPU usage.”article, stated
• Memory profiles are analyzed when runtime.mallocgc accounts for more than 15% of CPU time.
  “Additionally, if runtime.mallocgc—the Go runtime’s memory allocation function—accounts for more than 15% of CPU time, which means the runtime is spending a lot of time allocating memory, so PerfInsights also analyzes memory profiles to uncover potential allocation inefficiencies.”article, stated
• Validated detections dropped from 265 in February to 176 in June, a 33.5% reduction.
  “In February, we averaged 265 validated detections, reaching a single-day high of 500. By June, the figure had dropped to 176—a 33.5% reduction”article, stated
• The projected annual savings were about 3,800 hours of engineering time.
  “Addressing this volume manually, as the Go expert team would have consumed approximately 3,800 hours. By using our streamlined tool, we reduced the engineering time required to detect and fix an issue from 14.”article, stated
• Detection/fix effort dropped from 14.5 hours to almost 1 hour of tool runtime, a 93.10% time savings.
  “By using our streamlined tool, we reduced the engineering time required to detect and fix an issue from 14.5 hours to almost 1 hour of tool runtime—a 93.10% time savings.”article, stated

Alternatives considered (roads not taken)

Manual expert-led profiling and optimization

Rejected because: It required niche expertise and days, weeks, or months of effort that could not scale across services.

Single-shot LLM detection

Rejected because: It produced inconsistent results, hallucinations, and non-runnable generated code.

Trust a single model's detection judgment

Rejected because: Uber added LLM juries and LLMCheck because a single model can hallucinate or misread code constructs.

Patterns

eval-driven-development (evaluation)

PerfInsights uses LLMCheck metrics, dashboards, and validators to quantify detection accuracy, error patterns, and regressions.

llm-as-judge (evaluation)

A jury of LLMs independently validates whether each detected antipattern and optimization recommendation is valid.

guardrails-io-filtering (safety)

LLMCheck applies domain-specific rule validators to catch false positives such as loop invariant or map/slice misinterpretations.

prompt-versioning (prompt-operations)

Uber iteratively improved prompts with few-shot examples, audience/role framing, specific instructions, separate detection/validation prompts, and confidence scoring.

canary-rollout (operations)

The system earned adoption through early successful savings diffs before becoming part of the broader engineering toolkit.

Tradeoffs

Use profiling before LLM analysis

Chose runtime profile-driven candidate selection over scanning all Go code with an LLM.

Rationale: Profiling focuses the expensive language-model step on functions that matter for real compute cost.

Combine LLM juries with rule-based validation

Chose ensemble model judgment plus domain-specific validators over only one model or only static rules.

Rationale: The LLM can detect semantic antipatterns, while validators catch known false-positive classes and provide standardized metrics.

Embed optimization into continuous workflows

Chose CI/CD and day-to-day developer integration over reactive expert-led tuning only when systems hurt.

Rationale: Continuous surfacing makes performance tuning proactive and accessible to more teams.

Gotchas (what bit them)

Early LLM outputs were noisy and inconsistent.

Root cause: Single-shot prompts lacked enough task framing, examples, and validation for code-antipattern detection.

Resolution: Add few-shot examples, Go-expert role framing, clear positive instructions, separate detection/validation prompts, and confidence scoring.

LLMs hallucinated antipatterns and code semantics.

Root cause: Models could detect nonexistent antipatterns, confuse maps with slices, or misidentify loop invariants.

Resolution: Use LLM juries plus LLMCheck domain-specific validators to catch false positives before developer action.

Unfiltered code search space would overwhelm the detector and developers.

Root cause: A large monorepo contains dependencies, runtime internals, and low-value functions that are unlikely optimization targets.

Resolution: Filter aggressively to top expensive service-owned functions before GenAI analysis.

Evaluation

Uber evaluates PerfInsights through validated detection counts, LLMCheck false-positive metrics, hallucination-rate reduction, merged diffs, time-saved estimates, and codebase antipattern reduction over time.

Metrics: false positives reduced from over 80% to low teens · 33.5% reduction in validated detections over four months · 3,800 projected engineering hours saved annually · 93.10% time savings per issue · hundreds of merged diffs

Outcomes

• PerfInsights generated hundreds of merged diffs in Uber's Go monorepo.
• Validated antipattern detections fell by 33.5% over four months.
• Detection and fix time dropped from 14.5 hours to almost 1 hour of tool runtime.
• LLMCheck helped cut hallucination rates by more than 80%.
• The system shifted performance tuning from reactive expert work to continuous data-driven practice.

The system targets Uber's Go monorepo and the high-cost services whose top 10 accounted for more than multi-million dollars of compute spend in March 2024.