Observability Platform's Journey to Production GenAI Integration

new_relic · Tech · 2023 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Observability platform GenAI integration

New Relic needed to integrate generative AI into both internal developer workflows and customer-facing observability products at petabyte-scale data volume.

Constraints:

New Relic processes seven petabytes of observability data daily.
AI uses had to span code review, test generation, RCA, cost optimization, alerting, NRQL translation, and monitoring.
The team needed to choose between classical ML and generative AI depending on the task.
Cloud cost optimization involved financial impact at multimillion-dollar scale.

Architecture

New Relic used generative AI for developer productivity, multi-agent workflow automation, incident RCA, cloud cost optimization, AI monitoring, natural-language NRQL translation, alert optimization, and change tracking while retaining classical ML for forecasting-style tasks.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  developer-productivity-ai["Developer Productivity AI"]
  multi-agent-automation["Multi-Agent Automation"]
  incident-rca["Incident RCA"]
  cloud-cost-optimization["Cloud Cost Optimization"]
  observability-data-platform["Observability Data Platform"]
  nrql-translation["NRQL Translation"]
  ai-monitoring["AI Monitoring"]
  engineering-workflows["Engineering Workflows"]
  developer-productivity-ai --> engineering-workflows
  multi-agent-automation --> incident-rca
  multi-agent-automation --> cloud-cost-optimization
  incident-rca --> observability-data-platform
  cloud-cost-optimization --> observability-data-platform
  observability-data-platform --> nrql-translation
  observability-data-platform --> ai-monitoring
  nrql-translation --> observability-data-platform

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Developer Productivity AI`	Supports code reviews, test generation, and refactoring.	`Engineering Workflows`
`Multi-Agent Automation`	Coordinates workflow automation, incident management, RCA, and cost optimization.	`Incident RCA`, `Cloud Cost Optimization`
`Incident RCA`	Generates automated root-cause analysis for operations workflows.	`Observability Data Platform`
`Cloud Cost Optimization`	Manages convertible RI decisions to reduce cloud spend.	`Observability Data Platform`
`Observability Data Platform`	Processes observability data and powers customer-facing AI features.	`NRQL Translation`, `AI Monitoring`
`NRQL Translation`	Translates natural language into New Relic query language.	`Observability Data Platform`
`AI Monitoring`	Monitors AI systems and supports alert optimization and change tracking.	—

Data flow

Internal engineering and product signals flow into AI-assisted code and operations workflows; multi-agent systems automate incident and cost workflows; customer product features translate natural language into NRQL, monitor AI systems, optimize alerts, and track changes.

“They built a multi-agent architecture for workflow automation, incident management with automated RCA generation, and cloud cost optimization”summary, stated

“integrated AI monitoring capabilities, natural language query translation for their NRQL database language, automated alert optimization, and change tracking”summary, stated

Why this architecture

New Relic matched AI type to task: generative AI for language, code, RCA, and workflow automation, while retaining classical ML for forecasting and other suitable observability tasks.

“maintaining a balanced approach between classical machine learning for forecasting and generative AI for appropriate use cases.”summary, stated

“processing seven petabytes of observability data daily”summary, stated

Models used: Generative AI · Classical machine learning · Multi-agent architecture

Scale

New Relic processes seven petabytes of observability data daily.
“processing seven petabytes of observability data daily”summary, stated
Internal AI produced about 15% productivity gains across engineering levels.
“achieved approximately 15% productivity gains across engineering levels”summary, stated
Cloud cost optimization saved $4-5 million annually.
“saved $4-5 million annually through intelligent convertible RI management.”summary, stated

Alternatives considered (roads not taken)

Use generative AI for every observability task

Rejected because: Classical ML remained better suited for forecasting-style workloads.

“balanced approach between classical machine learning for forecasting and generative AI for appropriate use cases.”summary, stated

Limit GenAI to internal productivity

Rejected because: New Relic also integrated AI monitoring, NRQL translation, alert optimization, and change tracking into the product.

“externally within their product platform.”summary, stated

Patterns

`multi-agent-orchestration` (agent-architecture)

New Relic used a multi-agent architecture for workflow automation, RCA, and cost optimization.

“multi-agent architecture for workflow automation, incident management with automated RCA generation, and cloud cost optimization”summary, stated

`observability-tracing` (operations)

The platform added AI monitoring, alert optimization, and change tracking features.

“AI monitoring capabilities, natural language query translation for their NRQL database language, automated alert optimization, and change tracking”summary, stated

`query-rewriting` (query-generation)

New Relic translates natural language into NRQL.

“natural language query translation for their NRQL database language”summary, stated

`cost-aware-routing` (operations)

AI managed cloud-reservation decisions to reduce annual spend.

“saved $4-5 million annually through intelligent convertible RI management.”summary, stated

Tradeoffs

Use GenAI where language/workflow generation fits

Chose Code review, RCA, NRQL translation, alert optimization over Applying GenAI uniformly to forecasting.

Rationale: The team maintained a task-appropriate split between GenAI and classical ML.

“classical machine learning for forecasting and generative AI for appropriate use cases.”summary, stated

Use AI for cloud RI management

Chose Intelligent convertible RI management over Manual reservation optimization.

Rationale: The AI system delivered multimillion-dollar annual savings.

“cloud cost optimization that saved $4-5 million annually through intelligent convertible RI management.”summary, stated

Gotchas (what bit them)

Generative AI should not replace classical ML indiscriminately.

Root cause: Forecasting and other observability tasks may still fit traditional ML better.

Resolution: Use a balanced architecture that chooses classical ML or GenAI by task.

“balanced approach between classical machine learning for forecasting and generative AI for appropriate use cases.”summary, stated

AI observability must cover both internal development and product-facing operations.

Root cause: New Relic's GenAI usage spanned code productivity, RCA, cost optimization, monitoring, query translation, and alerts.

Resolution: Build internal and external AI integrations rather than a single feature.

“implemented generative AI both internally to improve developer productivity and externally within their product platform.”summary, stated

Evaluation

New Relic measured engineering productivity gains and annual cost savings, and framed AI product integrations around observability workflows.

Metrics: Engineering productivity gain · Annual cloud cost savings · Daily data volume

“achieved approximately 15% productivity gains across engineering levels”summary, stated

Outcomes

Internal AI delivered about 15% engineering productivity gains.
Cloud cost optimization saved $4-5 million annually.
Product AI features included AI monitoring, NRQL translation, alert optimization, and change tracking.

New Relic processes seven petabytes of observability data daily.