Production Monitoring and Issue Discovery for AI Agents

raindrop · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production monitoring and issue discovery for conversational AI agents

Raindrop addresses the problem that production agents fail silently and contextually: users behave unpredictably, failures often do not throw errors, and offline evals cannot cover multi-turn, tool-using, memory-bearing agents.

Constraints:

• Real-world user behavior cannot be fully anticipated before deployment.
• Agent failures often appear as poor behavior, refusals, forgetfulness, or frustration rather than exceptions.
• Issue tracking requires more precision than discovery because it drives counts, trends, and prioritization.
• Monitoring must cover semantic product quality, not only latency, token counts, and cost.

Architecture

Raindrop proposes a discover-track-fix loop for AI agents. Production conversations emit explicit signals such as thumbs-down, regeneration, copy, and upgrade actions, plus implicit semantic signals such as frustration, task failures, refusals, laziness, memory failures, and wins. The system clusters signals and intents, refines issue definitions with labeled examples, tracks frequency and dimensions over time, and then fixes high-impact issues through targeted prompts, guardrails, tool monitoring, or Trellis-style sub-agent/workflow decomposition.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  production-conversations["Production conversations"] --> explicit-signal-collectors["Explicit signal collectors"]
  production-conversations["Production conversations"] --> implicit-signal-detectors["Implicit signal detectors"]
  production-conversations["Production conversations"] --> tool-call-monitor["Tool-call monitor"]
  explicit-signal-collectors["Explicit signal collectors"] --> signal-and-intent-clustering["Signal and intent clustering"]
  implicit-signal-detectors["Implicit signal detectors"] --> signal-and-intent-clustering["Signal and intent clustering"]
  signal-and-intent-clustering["Signal and intent clustering"] --> issue-definitions["Issue definitions"]
  issue-definitions["Issue definitions"] --> deep-search-tracker["Deep Search tracker"]
  implicit-signal-detectors["Implicit signal detectors"] --> deep-search-tracker["Deep Search tracker"]
  deep-search-tracker["Deep Search tracker"] --> human-label-feedback["Human label feedback"]
  human-label-feedback["Human label feedback"] --> issue-definitions["Issue definitions"]
  deep-search-tracker["Deep Search tracker"] --> prioritization-dashboard["Prioritization dashboard"]
  tool-call-monitor["Tool-call monitor"] --> prioritization-dashboard["Prioritization dashboard"]
  prioritization-dashboard["Prioritization dashboard"] --> trellis-fix-workflow["Trellis fix workflow"]
  trellis-fix-workflow["Trellis fix workflow"] --> production-conversations["Production conversations"]

Components

Data flow

A production conversation is logged with user actions, agent outputs, tool calls, model metadata, and conversation context. Explicit and implicit signal detectors flag possible wins or failures. Semantic clustering groups those signals by issue and user intent. Deep Search turns an issue description into candidate matches, reranks and classifies them, then lets users label true/false matches so the definition improves. Tracking dashboards rank issues by users affected and dimensions such as models or tools, and fixes are applied through prompts, guardrails, or decomposition into specialized workflows/sub-agents.

Why this architecture

Raindrop separates discovery from tracking because loose clustering is useful for finding unknown issues, while dashboards need precise stable definitions. The architecture also separates operational telemetry from semantic quality monitoring: latency and token counts matter, but the central product question is whether the agent is actually helping users. Trellis then converts observed production intents into modular sub-agents so fixes do not destabilize unrelated behavior.

Models used: LLM binary classifiers · embedding models · LLM rerankers · lightweight trained models · Five Nano · GPT-4 context-free grammar outputs · structured output models

Scale

• Deep Research examples are described as 15-20 minute analyses.
  “Examples include Deep Research producing genuinely valuable 15-20 minute analyses”summary, stated
• Long-running agents may accumulate context across days, weeks, or months.
  “longer-running agents accumulate context over extended conversations that can span days, weeks, or months.”summary, stated
• Sentry-style prioritization distinguishes one-user errors from issues affecting eighty percent of users.
  “When Sentry shows an error that affected one user one time, developers typically ignore it. When the same error affects eighty percent of users, it becomes critical.”summary, stated
• Raindrop trains lightweight models so issue detection can process millions of events per day.
  “they train lightweight models under the hood so they can process millions of events per day without prohibitive costs.”summary, stated
• The Trellis framework was developed with Leaves, which has over six million users.
  “The most significant is what he calls the Trellis framework, developed in collaboration with Leaves, a studio with over six million users across multiple AI applications.”summary, stated

Alternatives considered (roads not taken)

Relying primarily on offline evals and unit-test-style cases

Rejected because: They cannot cover unpredictable real-world behavior, long contexts, memory, tool customizations, and undefined agent behavior.

General LLM-as-judge scoring for subjective quality

Rejected because: Subjective scoring is finicky and circular; binary detection of specific issue signals is more reliable.

Broad categories such as user frustration without subclusters

Rejected because: High-level categories do not tell teams which root cause to fix.

Keep all capabilities in one general agent prompt

Rejected because: A single agent becomes hard to change without breaking working use cases; high-frequency intents should become targeted sub-agents or workflows.

Patterns

observability-tracing (operations)

Raindrop monitors production conversations for semantic product failures, explicit feedback, tool-call behavior, and issue trends rather than only latency/cost telemetry.

eval-driven-development (quality)

The discover-track-fix loop turns production signals into issue definitions, tracked metrics, and experiments that guide agent improvements.

multi-agent-orchestration (architecture)

The Trellis guidance treats tools as specialized sub-agents and moves high-frequency intents out of the general agent into targeted workflows.

tool-use-orchestration (architecture)

Raindrop recommends monitoring which tools are invoked, whether they fail, and how often they repeat within conversations.

agent-memory (state)

Memory failures are treated as a first-class implicit signal for long-running agents that maintain and compress context over time.

Tradeoffs

Use production monitoring alongside offline evals

Chose Discover-track-fix from live conversations over Only predefined offline test cases.

Rationale: Production monitoring finds emergent user intents and silent failures that offline test suites cannot anticipate.

Detect specific implicit signals

Chose Binary issue detection for well-defined signals over General subjective quality scoring with an LLM judge.

Rationale: Specific binary detection is more reliable and cheaper to refine than asking models to grade broad subjective qualities.

Accept loose discovery but demand precise tracking

Chose False-positive-tolerant clustering followed by refined issue definitions over One precision requirement for every stage.

Rationale: Humans can review noisy discovery clusters, but trend dashboards need stable and accurate counting.

Modularize high-frequency functionality

Chose Specialized sub-agents and workflows over A single all-purpose agent prompt.

Rationale: Modularity lets teams improve a capability without destabilizing unrelated behaviors.

Gotchas (what bit them)

AI agents can fail without emitting conventional errors.

Root cause: Failures manifest as hallucinated promises, refusals, laziness, forgetfulness, or user frustration rather than exceptions.

Resolution: Monitor semantic explicit and implicit signals in production conversations.

Offline evaluations cannot anticipate the full state space of deployed agents.

Root cause: Multi-turn agents accumulate long context, memory, enabled/disabled tools, and user-specific workflows over time.

Resolution: Combine offline tests with production monitoring, issue discovery, and continuous refinement.

Broad issue labels do not produce actionable fixes.

Root cause: Categories like user frustration hide multiple unrelated sub-causes, intents, and product contexts.

Resolution: Cluster signals into subissues and combine them with intent discovery.

Tracking definitions can drift or overcount if they are not refined.

Root cause: Issue descriptions initially mix false positives and neighboring failure modes such as memory failure versus hallucination.

Resolution: Let humans label correct/incorrect examples and feed those labels back into clarifying issue definitions.

One general prompt becomes risky to change once users depend on it.

Root cause: Prompt modifications can improve one use case while breaking already-working behavior elsewhere.

Resolution: Discretize high-frequency intents into specialized sub-agents or workflows and improve them separately.

Evaluation

Raindrop evaluates agent quality through production issue signals, semantic clustering, intent pairing, Deep Search candidate classification, human correction of matches, dimensional monitoring, and experiments that verify whether fixes reduce tracked issues.

Metrics: thumbs up/down · regeneration rate · copied response portions · upgrade actions · refusals · task failures · laziness · memory failures · user frustration · issue frequency over time · affected users · tool-call failures · model and tool dimensions

Outcomes

• Raindrop provides a concrete production-monitoring framework for discovering, tracking, and fixing AI-agent issues.
• The Trellis framework turns observed production intents into specialized workflows or sub-agents.
• Deep Search lets teams refine issue definitions and track patterns against production conversations.
• The presentation reframes AI reliability as an experimentation and monitoring discipline rather than a one-time eval suite.

The strongest scale references are Leaves' six million users, millions of events per day for lightweight issue detection, and conversations that span days, weeks, or months.