PerfInsights: AI-Powered Performance Optimization for Go Services

uber · Tech · 2025 · source Agentic: yes · promotional bias: high · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Performance optimization for large-scale Go services

Uber needed to reduce expensive compute usage in Go backend services, but traditional profiling and optimization required specialized Go/runtime expertise and too much manual effort to scale across teams.

Constraints:

• The top 10 Go services alone represented multi-million-dollar monthly compute spend.
• Manual Go performance tuning could take days, weeks, or months and required compiler/runtime expertise.
• A naive LLM detector produced inconsistent outputs, hallucinations, and non-runnable code.
• Developers needed high precision and trust before accepting AI-suggested optimizations.
• The system had to focus on service-owned hotpaths rather than noisy dependencies or runtime internals.
• Suggestions needed to flow into existing optimization, CI/CD, and developer workflows.

Architecture

PerfInsights combines deterministic production profiling with GenAI static analysis. Uber's daily fleet-wide profiler identifies hot Go functions during peak traffic, static filters remove open-source and runtime code, an antipattern catalog guides an LLM detector over full source code, and two validation layers - LLM juries plus rule-based LLMCheck validators - reduce hallucinations. Validated findings produce confidence-scored optimization suggestions that flow into Optix, dashboards, CI/CD, manual review, and continuous code optimization workflows.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  Fleet_wide_profiler["Fleet-wide profiler"] --> Hotpath_function_filter["Hotpath function filter"]
  Hotpath_function_filter --> Static_ownership_filter["Static ownership filter"]
  Static_ownership_filter --> Antipattern_catalog["Antipattern catalog"]
  Static_ownership_filter --> LLM_detector["LLM detector"]
  Antipattern_catalog --> LLM_detector
  LLM_detector --> LLM_jury["LLM jury"]
  LLM_detector --> LLMCheck_validators["LLMCheck validators"]
  LLM_jury --> LLMCheck_validators
  LLMCheck_validators --> Metrics_dashboard["Metrics dashboard"]
  LLMCheck_validators --> Optix_integration["Optix integration"]
  Metrics_dashboard --> Developer_review["Developer review"]
  Optix_integration --> Developer_review
  Optix_integration --> CI_CD_workflow["CI/CD workflow"]
  Developer_review --> CI_CD_workflow

Components

Data flow

Production CPU and memory profiles identify the top flat-CPU functions and allocation-heavy services; static filters keep only service-owned candidate code; the LLM receives full source code plus antipattern definitions; LLM juries independently validate detections; LLMCheck applies domain-specific validators and logs accuracy/drift metrics; accepted suggestions and confidence scores feed downstream code transformation or developer review through Optix and CI/CD.

Why this architecture

PerfInsights constrains the LLM with production profiling, service ownership filters, and a Go-specific antipattern catalog because raw single-shot LLM analysis was unreliable. The architecture layers ensemble and deterministic validation because developer trust and low false positives are prerequisites for code-optimization adoption.

Models used: Large language model · LLM juries · LLMCheck · Optix

Scale

• In March 2024, Uber's top 10 Go services accounted for more than multi-million dollars in monthly compute spend.
  “In March 2024 alone, the top 10 Go services accounted for more than multi-million dollars in compute spend alone”article, stated
• PerfInsights analyzes the top 30 flat-CPU functions per service.
  “For each service, it identifies the top 30 most expensive functions based on flat CPU usage.”article, stated
• Memory profile analysis triggers when runtime.mallocgc exceeds 15% of CPU time.
  “if runtime.mallocgc—the Go runtime’s memory allocation function—accounts for more than 15% of CPU time”article, stated
• False positives reportedly dropped from over 80% to the low teens.
  “reducing false positives from over 80% to the low teens.”article, stated
• Validated detections averaged 265 in February, peaked at 500 in one day, and dropped to 176 by June.
  “In February, we averaged 265 validated detections, reaching a single-day high of 500. By June, the figure had dropped to 176”article, stated
• The validated antipattern count dropped 33.5% in four months.
  “By June, the figure had dropped to 176—a 33.5% reduction in just four months.”article, stated
• The annualized reduction is projected at 267 antipatterns and approximately 3,800 manual engineering hours.
  “Projecting this annually, that’s a reduction of 267 antipatterns. Addressing this volume manually, as the Go expert team would have consumed approximately 3,800 hours.”article, stated
• The tool reportedly reduced detection-and-fix time from 14.5 hours to almost 1 hour, a 93.10% time savings.
  “we reduced the engineering time required to detect and fix an issue from 14.5 hours to almost 1 hour of tool runtime—a 93.10% time savings”article, stated
• Manual historical efforts cited include 320 hours, 160 hours, and 960 hours on specialized optimization work.
  “5 critical antipatterns once required two engineers (including a Principal Engineer) for a full month (∼320 hours).”article, stated

  “A dedicated Go expert spent six months full-time on related optimization projects (∼960 hours).”article, stated

Alternatives considered (roads not taken)

Analyze whole codebases with LLMs directly

Rejected because: PerfInsights first uses production profiles and static filters to reduce noise and focus LLM analysis on service-owned hotpath functions.

Trust a single-shot LLM detection result

Rejected because: Early single-shot detection was inconsistent, hallucinated, and generated non-runnable code.

Use one broad prompt for all antipatterns

Rejected because: The team found one prompt per antipattern preserved context and simplified tasks.

Use LLM juries without deterministic validators

Rejected because: Even juries can hallucinate, so LLMCheck adds domain-specific rule-based validators.

Keep optimization as reactive expert-led investigation

Rejected because: PerfInsights embeds optimization opportunities into CI/CD and developer workflows as a proactive loop.

Patterns

observability-tracing (operations)

PerfInsights starts from production CPU/memory profiles and uses LLMCheck dashboards to track accuracy, error patterns, antipattern frequency, and model regressions.

eval-driven-development (evaluation)

The system uses LLM juries, LLMCheck, confidence scores, prompt tuning metrics, and dashboards to validate LLM detections before developers act on them.

tool-use-orchestration (workflow)

Validated LLM findings flow into Optix, code transformation or manual review, CI/CD pipelines, and day-to-day developer workflows.

human-in-the-loop-gate (review)

PerfInsights outputs actionable recommendations and diffs into workflows where developers can manually review suggestions and build trust from dashboards.

prompt-versioning (prompt-operations)

The team iteratively tuned prompts with few-shot examples, Go-expert role framing, output-quality instructions, confidence scoring, and separate detection/validation prompts.

cost-aware-routing (optimization)

The system targets high-cost service hotpaths first, using CPU allocation profiles to focus LLM work on code that can reduce compute spend.

Tradeoffs

Pre-filter with production profiles before LLM inference.

Chose Deterministic hotpath and ownership filtering over LLM analysis of broad code surfaces.

Rationale: Profiling focuses LLM cost and attention on high-impact functions and prevents noisy suggestions on code teams do not own.

Use LLM juries plus rule validators.

Chose Hybrid ensemble and deterministic validation over Single-model judgment or LLM-only validation.

Rationale: The system needs LLM flexibility to reason about code but deterministic validators to catch known false-positive classes.

Spend more inference on one prompt per antipattern.

Chose Focused prompts over One broad multi-antipattern prompt.

Rationale: Focused prompts simplify the task and conserve useful context, trading more calls for better precision.

Integrate with Optix and CI/CD rather than ship a standalone dashboard-only tool.

Chose Embedded continuous optimization workflow over Manual one-off performance investigation.

Rationale: Embedding suggestions in existing code transformation and developer workflows makes performance tuning proactive and repeatable.

Ask the model for confidence scores.

Chose Confidence-scored recommendations over Unqualified optimization claims.

Rationale: Confidence requests encourage more thoughtful LLM responses and help downstream review triage.

Gotchas (what bit them)

Single-shot LLM detection was unreliable.

Root cause: The model varied between runs, hallucinated antipatterns, and produced non-runnable code when asked to detect issues directly.

Resolution: Improve prompts, score only detected antipatterns, split detection and validation, use few-shot examples, and add LLM juries plus LLMCheck.

The LLM falsely detected case-insensitive string comparison antipatterns.

Root cause: The prompt lacked examples clarifying when the EqualFold antipattern did and did not apply.

Resolution: Add few-shot examples, including cases that resolve the confusion.

LLMs can still hallucinate after an ensemble check.

Root cause: LLM juries reduce but do not eliminate errors such as nonexistent antipatterns, map/slice confusion, and loop-invariant mistakes.

Resolution: Run LLMCheck domain-specific validators after the jury.

Unfiltered code analysis overwhelms developers with noise.

Root cause: LLMs would otherwise inspect irrelevant dependencies, runtime internals, or cold code that teams cannot or should not optimize.

Resolution: Use production profiles and static filters to isolate hot service-owned code before LLM inference.

Developer trust is hard without measurable validation.

Root cause: Engineers need evidence that AI recommendations are accurate and improving, especially for code optimization.

Resolution: Expose detection accuracy, false-positive reasons, model regressions, and antipattern frequency through LLMCheck dashboards.

Evaluation

PerfInsights evaluates both code and model behavior: production profiling selects candidates, LLM juries validate antipattern existence and suggested fixes, LLMCheck rule validators catch false positives, confidence scores rank suggestions, dashboards track detection accuracy and regressions, and downstream Optix/CI/CD workflows validate code improvements.

Metrics: Flat CPU usage · runtime.mallocgc CPU share · Validated detections · False positive rate · Detection accuracy · Failure rates · Model drift · Confidence score · Antipattern frequency · Merged diffs · Engineering time saved · Hallucination rate · Compute cost savings

Outcomes

• PerfInsights reportedly turns Go performance optimization from days or weeks into hours.
• The validation strategy reduced false positives from over 80% to the low teens.
• The system produced hundreds of merged diffs in Uber's Go monorepo.
• Validated detections declined from 265 in February to 176 in June, a 33.5% reduction over four months.
• Uber reports an estimated 3,800 annual engineering hours avoided and a 93.10% reduction in detection-and-fix time.
• LLMCheck dashboards helped reduce hallucination rates by more than 80% and build developer trust.

Uber reports multi-million-dollar monthly compute spend for the top 10 Go services, daily fleet-wide profiling at peak traffic, top-30-function filtering, >15% runtime.mallocgc memory-analysis trigger, hundreds of merged diffs, and projected thousands of annual engineering hours saved.