Automated Log Classification System for Device Security Infrastructure

palo_alto_networks · Tech · 2026 · source Agentic: no · promotional bias: high · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Device security infrastructure operations

Palo Alto Networks' Device Security team needed to detect production issues earlier in more than 200 million daily service and application logs without sending every log through an expensive LLM classifier.

Constraints:

• daily log volume exceeds 200 million entries
• reactive processing delayed response to critical issues
• most logs are duplicate events with different timestamps or phrasing
• SMEs need explanations before acting on severity predictions
• the system must integrate with existing FluentD, Kafka, S3, and Redshift pipelines

Architecture

Palo Alto Networks and AWS GenAIIC built a three-stage Bedrock pipeline. FluentD and Kafka ingest logs. Stage 1 uses Aurora exact matching, overlap similarity, and Titan Text Embeddings semantic matching to deduplicate and cache over 99% of daily logs before LLM invocation. Stage 2 uses Titan embeddings and vector similarity search to retrieve labeled historical examples for the small set of unique logs. Stage 3 sends the unique log plus dynamically selected examples to Anthropic Claude Haiku on Amazon Bedrock, which returns structured P1/P2/P3 severity classifications and reasoning. Results are stored in Aurora/cache and flow to S3, Redshift, and SME review for action and continuous improvement.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  fluentd-and-kafka-ingestion[FluentD and Kafka ingestion] --> aurora-cache[Aurora cache]
  aurora-cache --> overlap-similarity-matcher[Overlap similarity matcher]
  overlap-similarity-matcher --> titan-semantic-matcher[Titan semantic matcher]
  titan-semantic-matcher --> historical-example-retriever[Historical example retriever]
  historical-example-retriever --> claude-haiku-classifier[Claude Haiku classifier]
  claude-haiku-classifier --> output-storage[Output storage]
  output-storage --> aurora-cache
  output-storage --> sme-review-interface[SME review interface]
  sme-review-interface --> historical-example-retriever

Components

Data flow

Incoming logs flow from FluentD and Kafka into an Aurora-backed cache. The system attempts exact matching, overlap similarity, and semantic similarity to reuse prior classifications. Unique logs pass to a Titan-embedding context retrieval stage that finds similar labeled examples. Bedrock with Claude Haiku classifies severity and reasoning, then persists outputs to Aurora/cache and downstream S3/Redshift reporting plus SME review.

Why this architecture

The design solves the scale problem by making the LLM the last resort rather than the first step. Exact, overlap, and semantic cache checks handle duplicate events cheaply, while dynamic example retrieval gives the LLM context only for truly unique logs. Claude Haiku returns structured severity plus reasoning so SMEs can act quickly, and every SME validation enriches the future few-shot dataset and cache.

Models used: Anthropic Claude Haiku · Amazon Titan Text Embeddings · Amazon Bedrock

Scale

• The pipeline processes over 200 million service and application logs daily.
  “The solution processes over 200 million service and application logs daily, automatically identifying critical issues before they escalate into service outages that impact customers.”article, stated
• Caching reduced the 200 million daily logs by over 99% to unique events.
  “This approach cost-effectively reduces the 200 million daily logs by over 99%, to logs only representing unique events.”article, stated
• The system achieved 95% precision and 90% recall for P1 severity logs.
  “* 95% precision, 90% recall for P1 severity logs – Critical alerts are accurate and actionable, minimizing false alarms while catching 9 out of 10 urgent issues, leaving the remaining alerts to be captured by existing monitoring systems”article, stated
• Debugging time was reduced by 83%.
  “* 83% reduction in debugging time – SMEs spend less time on routine log analysis and more time on strategic improvements”article, stated
• Cached results are served with subsecond responses.
  “* Over 99% cache hit rate – The intelligent caching layer processes 20 million daily volume cost-effectively through subsecond responses”article, stated

Alternatives considered (roads not taken)

Classify every log entry with an LLM

Rejected because: The team needed to avoid redundant LLM calls across hundreds of millions of mostly duplicate logs.

Static few-shot examples

Rejected because: Dynamic retrieval provides examples tailored to each unique log and evolving pattern.

Traditional rule-based log handling alone

Rejected because: The article says evolving log patterns are difficult for traditional rules, while dynamic examples can adapt as SMEs label new cases.

Patterns

semantic-caching (retrieval)

The system uses exact, overlap, and semantic matching to reuse prior classifications for duplicate log events before invoking an LLM.

rag-retrieval (retrieval)

Unique logs retrieve dynamically similar labeled historical examples to guide severity classification.

structured-output-contract (output-control)

Claude Haiku outputs P1/P2/P3 severity classifications plus detailed reasoning for SME action.

human-in-the-loop-gate (governance)

SME validations become new labeled examples and improve future classifications.

cost-aware-routing (operations)

The architecture routes common duplicate logs through cache and reserves LLM classification for unique logs.

Tradeoffs

Cache and deduplicate before LLM classification

Chose multi-layer caching and similarity matching over classifying every log directly.

Rationale: Most logs were duplicate events, so cache routing made real-time enterprise-scale LLM use economically viable.

Use dynamic few-shot retrieval

Chose vector-selected labeled examples over static prompt examples.

Rationale: Dynamic examples adapt to new log patterns and Palo Alto's internal systems.

Return severity reasoning with every classification

Chose structured severity plus explanation over bare P1/P2/P3 labels.

Rationale: SMEs needed to quickly validate AI decisions before taking preventive action.

Gotchas (what bit them)

LLM-per-log classification is economically infeasible at this volume.

Root cause: More than 200 million daily logs contain many duplicate events with superficial timestamp, level, or wording differences.

Resolution: Use exact, overlap, and semantic cache hits before classifying only the unique remainder.

Static examples can become stale as log patterns evolve.

Root cause: New categories and patterns appear over time in production infrastructure.

Resolution: Let SMEs label additional examples and retrieve them dynamically without code changes.

SMEs need trustable explanations for critical-severity alerts.

Root cause: P1 classifications guide urgent operational action, so unexplained predictions are not enough.

Resolution: Emit detailed reasoning with each severity classification.

Evaluation

Palo Alto evaluates the system with production-oriented precision/recall, debugging-time reduction, cache-hit rate, SME validation, and continuous improvement from newly labeled examples.

Metrics: 95% precision for P1 severity logs · 90% recall for P1 severity logs · 83% reduction in debugging time · over 99% cache hit rate · cache efficiency improvement from SME validation

Outcomes

• The solution achieved 95% precision and 90% recall for P1 severity logs.
• Debugging time was reduced by 83%.
• Over 99% of logs were handled through cache hits.
• The system shifted operations from reactive monitoring toward proactive issue detection.

The system is designed for more than 200 million daily logs and integrates with Palo Alto Networks' existing production log pipeline.