Optimizing Security Threat Investigation with Multi-Model LLM Strategy

trellix · Tech · 2025 · source Agentic: yes · promotional bias: high · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Cybersecurity threat investigation cost and latency optimization

Trellix needed to reduce the cost and latency of Trellix Wise threat investigations without losing the completeness and reliability required for automated security-event analysis and risk scoring.

Constraints:

• Security teams face expanding threats while constrained by talent and budget.
• Trellix Wise must process security-event investigations fast enough to support analyst triage.
• The data-collection stage needed completeness while minimizing hallucinations.
• Not every workflow stage required the cost and accuracy profile of Claude Sonnet.
• The new model strategy needed validation in a use-case-specific test harness and phased production rollout.

Architecture

Trellix Wise runs threat investigations on Amazon Bedrock over security-event data stored in Amazon OpenSearch Service. OpenSearch provides vector-database context through Bedrock Knowledge Bases; the workflow retrieves event-related data, analyzes it with foundation models and custom ML insights, assigns a risk score, and presents results in a dashboard. Trellix introduced Amazon Nova Micro for lower-cost multi-pass data collection while retaining Claude Sonnet where stronger capability is needed.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  Security_events["Security events"] --> Amazon_OpenSearch_Service["Amazon OpenSearch Service"]
  Amazon_OpenSearch_Service --> Amazon_Bedrock_Knowledge_Bases["Amazon Bedrock Knowledge Bases"]
  Amazon_Bedrock_Knowledge_Bases --> Amazon_Nova_Micro["Amazon Nova Micro"]
  Amazon_Bedrock_Knowledge_Bases --> Claude_Sonnet["Claude Sonnet"]
  Amazon_Nova_Micro --> Threat_investigation_workflow["Threat investigation workflow"]
  Claude_Sonnet --> Threat_investigation_workflow
  Threat_investigation_workflow --> Custom_ML_models["Custom ML models"]
  Custom_ML_models --> Threat_investigation_workflow
  Threat_investigation_workflow --> Risk_scoring["Risk scoring"]
  Risk_scoring --> Trellix_Wise_dashboard["Trellix Wise dashboard"]
  Threat_investigation_workflow --> Trellix_Wise_dashboard
  Trellix_Wise_dashboard --> Security_analysts["Security analysts"]

Components

Data flow

Security events are stored in OpenSearch Service, retrieved as RAG context through Bedrock Knowledge Bases, collected through one or more Bedrock inference calls, analyzed with custom ML insights, scored for risk, and shown in the Trellix Wise dashboard so analysts can decide whether human follow-up is necessary.

Why this architecture

Trellix split the workflow by task requirements: RAG grounds the security investigation, Claude Sonnet remains available as the primary model, and Nova Micro handles the data-collection stage where multiple cheaper, faster passes can improve completeness at lower cost. A custom test harness and phased pilot reduce risk before broader production rollout.

Models used: Anthropic Claude Sonnet · Amazon Nova Micro · Amazon Nova Lite

Scale

• Trellix serves over 53,000 customers worldwide.
  “Trellix, a leading company delivering cybersecurity’s broadest AI-powered platform to over 53,000 customers worldwide”article, stated
• Security-event investigations that previously required multiple analysts and hours can now complete in seconds.
  “security teams can now complete what used to take multiple analysts hours of work to investigate in seconds”article, stated
• OpenSearch stores billions of security events for Trellix Wise.
  “The platform uses the Amazon OpenSearch Service stores billions of security events collected from the environments monitored.”article, stated
• Amazon Nova Micro delivered inferences three times faster in early evaluations.
  “Amazon Nova Micro delivered inferences three times faster and at nearly 100-fold lower cost.”article, stated
• Amazon Nova Micro's cost per inference was nearly 100 times lower.
  “Its speed allows it to process 3-5 inferences in the same time as a single Claude Sonnet inference and it’s cost per inference is almost 100 times lower.”article, stated
• Running multiple Nova Micro inferences still lowered costs by a factor of 30.
  “by running multiple inferences, you can maximize the coverage of required data and still lower costs by a factor of 30.”article, stated

Alternatives considered (roads not taken)

Use Claude Sonnet for all stages

Rejected because: Some investigation stages did not need Claude Sonnet's accuracy and could benefit from faster, lower-cost models.

Rely on a single small-model inference for data collection

Rejected because: Multiple Nova Micro passes improve data coverage while still reducing cost.

Evaluate models using only standardized benchmarks

Rejected because: Trellix judged that standardized benchmarks were insufficient for their specific investigation workflow.

Immediate broad production rollout

Rejected because: The approach was deployed first in a limited pilot with detailed evaluations during phased rollout.

Patterns

model-routing (model-operations)

Trellix assigns different workflow stages to Claude Sonnet or Amazon Nova Micro based on capability, speed, and cost requirements.

cost-aware-routing (model-operations)

The data collection stage runs multiple cheaper Nova Micro inferences to reduce overall investigation cost.

rag-retrieval (retrieval)

OpenSearch vector data is used as context for Bedrock Knowledge Bases in the threat investigation workflow.

eval-driven-development (quality)

Trellix used a workflow-specific test harness to evaluate response completeness, cost, and speed before rollout.

safety-policy-enforcement (safety)

Prompt engineering and reference-data scaffolding constrain Nova Micro responses to limit hallucinations.

canary-rollout (deployment)

The new multi-model approach is deployed in a limited pilot before phased production rollout.

Tradeoffs

Use multiple Nova Micro passes instead of one Claude Sonnet call for data collection.

Chose Repeated small-model inferences over Single larger-model inference.

Rationale: Multiple Nova Micro calls improve coverage while lowering cost and increasing speed.

Optimize the data-collection stage separately from the rest of the investigation workflow.

Chose Stage-specific model strategy over One-model-for-all-stages architecture.

Rationale: The data-collection stage was the optimal place to optimize speed and cost.

Use custom workflow evaluation instead of relying on standardized benchmarks.

Chose Use-case-specific test harness over Standardized benchmark-only evaluation.

Rationale: The team needed to validate completeness, cost, and speed for their actual information-gathering workflow.

Limit rollout before full production adoption.

Chose Limited pilot and phased rollout over Immediate broad production rollout.

Rationale: The team wanted continued detailed evaluation before wider production deployment.

Gotchas (what bit them)

The most capable model is not always the most cost-effective choice.

Root cause: Different investigation stages have different accuracy, cost, and latency requirements.

Resolution: Route cheaper Nova Micro to the data-collection stage while retaining Claude Sonnet where needed.

Smaller-model responses were more variable.

Root cause: Nova Micro had higher variability than larger models in Trellix's evaluations.

Resolution: Run multiple passes to produce a more exhaustive response set.

Generic model benchmarks did not capture production workflow fit.

Root cause: The target use case had specific information-gathering requirements.

Resolution: Build a test harness replicating Trellix's information-gathering workflows.

Completeness and hallucination risk pull in opposite directions during retrieval data collection.

Root cause: The data-collection stage must maximize required data while minimizing inaccurate model output.

Resolution: Constrain responses with proprietary prompt engineering and reference data.

Evaluation

Trellix built a test harness that reproduced its information-gathering workflows and measured response completeness, cost, and inference speed before limited pilot deployment.

Metrics: Response completeness · Cost per inference · Speed of inference · 3-5 Nova Micro inferences per single Claude Sonnet inference time · Almost 100x lower cost per inference · 30x lower data-collection cost

Outcomes

• Amazon Nova Micro delivered inferences three times faster in early evaluations.
• Cost per inference was nearly 100 times lower.
• Multiple Nova Micro passes reduced data-collection costs by a factor of 30 while increasing coverage.
• The approach entered a limited pilot with detailed evaluations for phased production rollout.

Trellix Wise serves a cybersecurity platform for over 53,000 customers, stores billions of security events in OpenSearch Service, and targets investigations that previously took multiple analysts hours but can now complete in seconds.