Panel Discussion on AI Agents in Production: Security, Evaluation, and Infrastructure

zenity_/hetz/aidoc/band/_mongodb · Tech · 2026 · source Agentic: yes · promotional bias: med · summary-only (no fetched source) · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production AI-agent security, evaluation, and infrastructure

The panel describes how organizations deploy AI agents in production while managing non-determinism, context quality, security boundaries, high-stakes evaluation, multi-agent communication, and flexible data-layer infrastructure.

Constraints:

• Agents are non-deterministic and may combine tools in unanticipated ways.
• Prompt-based restrictions are not sufficient security controls.
• High-stakes medical AI requires hospital-level evaluation, drift monitoring, and false-positive/false-negative control.
• Multi-agent systems need identity, communication, federation, retries, and tracing across heterogeneous frameworks.
• Context and data layers must support vector search, text search, standard queries, state, security, monitoring, and event reactivity.
• This extraction is summary-only because the fetched transcript was not usable.

Architecture

The panel's production architecture combines context management, deterministic security controls, evaluation infrastructure, operations monitoring, multi-agent communication infrastructure, and a flexible data layer. Agents receive curated context and scoped tools; permissions are narrowed per execution; traces and monitoring observe every action; high-stakes models pass multi-layer evaluation and AI operations review; autonomous agents communicate through infrastructure rather than manual copy/paste; and databases provide memory, search, security, and event-driven reactivity.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  Context_management_layer["Context management layer"] --> Agent_execution_runtime["Agent execution runtime"]
  Data_layer["Data layer"] --> Context_management_layer
  Data_layer --> Agent_execution_runtime
  Agent_execution_runtime --> Deterministic_security_controls["Deterministic security controls"]
  Deterministic_security_controls --> Sandbox_or_isolated_environment["Sandbox or isolated environment"]
  Sandbox_or_isolated_environment --> Agent_execution_runtime
  Agent_execution_runtime --> Observability_traces["Observability traces"]
  Observability_traces --> AI_operations_center["AI operations center"]
  Observability_traces --> Evaluation_infrastructure["Evaluation infrastructure"]
  Evaluation_infrastructure --> AI_operations_center
  Evaluation_infrastructure --> Customer_analytics["Customer analytics"]
  Agent_execution_runtime --> Multi_agent_communication_layer["Multi-agent communication layer"]
  Multi_agent_communication_layer --> Agent_execution_runtime

Components

Data flow

Organizational knowledge is organized into a context/data layer, retrieved through vector/text/standard queries, supplied to agents under deterministic permissions, monitored through traces and operations dashboards, evaluated against development and production datasets, and communicated across multi-agent infrastructure when agents need to coordinate.

Why this architecture

The panel argues that production agent systems need deterministic infrastructure around non-deterministic models: context makes models useful, scoped permissions and sandboxes limit harm, traces make behavior visible, high-stakes domains require evaluation-heavy release processes, and multi-agent systems need communication infrastructure instead of ad hoc manual coordination.

Models used: AI agents · Proprietary frontier medical models · External LLMs · Open-weight models · API-based models

Scale

• Aidoc is deployed in approximately 2,000 hospitals globally.
  “The system is deployed in approximately 2,000 hospitals globally and has raised $500 million in total funding.”summary, stated
• Aidoc has raised $500 million in total funding.
  “The system is deployed in approximately 2,000 hospitals globally and has raised $500 million in total funding.”summary, stated
• Aidoc dedicates 20-30% of development effort for each new version to evaluation.
  “with 20-30% of development effort for each new version dedicated solely to evaluation.”summary, stated
• Aidoc evaluates tens of millions of scans weekly.
  “Aidoc performs evaluations on tens of millions of scans weekly”summary, stated
• Some medical conditions appear in 10% or less of data.
  “Medical problems are often extremely rare, with some conditions appearing in only 10% or less of data.”summary, stated
• Aidoc increased output from about three algorithms per year to 15 in the past year.
  “enabled them to increase their output from developing approximately three algorithms per year to 15 algorithms in the past year alone”summary, stated
• The panel discusses production concerns for hundreds of agents.
  “Beyond these infrastructure challenges, organizations face additional problems when running hundreds of agents in production.”summary, stated

Alternatives considered (roads not taken)

Rely on prompt instructions for security

Rejected because: The panel says prompt-based controls do not work as genuine security mechanisms for agents.

Grant broad database access and trust code or prompts to filter

Rejected because: Agents may choose unexpected access paths, so permissions should be scoped to the execution and customer data needed.

Use generic frontier models for all medical imaging cases

Rejected because: Medical imaging data and rare conditions require proprietary data to reach necessary accuracy levels.

Manual copy/paste communication between agent sessions

Rejected because: Band argues autonomous agents should communicate directly rather than forcing humans to act as the communication layer.

Treat vector search as a universal solution

Rejected because: Vector search is useful for semantic similarity but may be wrong when structured queries or other data access patterns are needed.

Patterns

context-engineering (context)

The panel frames context management as the primary success factor for production agents, translating institutional knowledge into usable model context.

safety-policy-enforcement (safety)

Agent permissions are narrowed to exact execution needs, with deterministic boundaries instead of prompt-only rules.

observability-tracing (operations)

Teams should trace everything agents do in production for incident response and continuous improvement.

eval-driven-development (evaluation)

Aidoc dedicates a large share of development effort to release evaluation, hospital-level metrics, drift checks, and operations monitoring.

multi-agent-orchestration (orchestration)

The panel describes orchestrator/sub-agent patterns and future autonomous agents communicating bidirectionally across frameworks and languages.

agent-memory (state)

The MongoDB discussion emphasizes memory architecture, state, and data-layer support beyond one-shot retrieval.

rag-retrieval (retrieval)

Vector search, text search, and standard queries are treated as tools agents may use to retrieve the right context.

Tradeoffs

Secure agents with deterministic boundaries.

Chose Scoped permissions and isolated execution over Prompt instructions as security controls.

Rationale: Agents can choose unexpected actions, so enforceable controls must prevent unauthorized access regardless of the agent's reasoning.

Train proprietary medical models.

Chose Large proprietary medical datasets and in-house frontier models over Generic closed-source frontier models.

Rationale: Medical imaging has rare conditions and extreme accuracy requirements that generic internet-trained models cannot reliably cover.

Use a flexible data layer instead of many specialized systems.

Chose One infrastructure layer with vector search, text search, standard queries, state, security, monitoring, and events over Agents accessing many disparate systems.

Rationale: Combining essential capabilities reduces developer and agent complexity while preserving flexibility for changing AI architectures.

Gotchas (what bit them)

Prompt-only security does not constrain non-deterministic agents.

Root cause: Agents can generate execution paths and combine tools in ways developers did not anticipate.

Resolution: Add deterministic permission scoping, capability audits, isolation, tracing, and hooks.

Agents often have unnecessary capabilities.

Root cause: Teams connect MCP servers, tools, frameworks, or terminal execution without auditing need.

Resolution: Audit what agents can do and remove unnecessary capabilities to reduce attack surface.

High-stakes model improvement can hide local regressions.

Root cause: Aggregate metrics may improve while specific hospitals or data slices degrade.

Resolution: Evaluate at individual hospital level and investigate localized degradation before release.

Multi-agent production becomes a distributed-systems problem.

Root cause: Autonomous agents may run continuously across frameworks, languages, harnesses, identities, and environments.

Resolution: Provide infrastructure for discovery, tracing, identity, retries, federation, logs, and conversations.

Wrong context compounds agent errors.

Root cause: Non-deterministic agents build on retrieved context; irrelevant retrieval can send work in the wrong direction.

Resolution: Choose retrieval/search/query tools based on data shape and use case rather than defaulting to vector search.

Evaluation

The panel presents evaluation as a multi-layer practice: development test sets reviewed by radiologists, fresh production-data snapshots from hospitals, false-negative and false-positive metrics at individual hospital level, continuous AI operations monitoring, automated annotation infrastructure, and customer-facing analytics.

Metrics: 20-30% of development effort dedicated to evaluation · Standardized radiologist-reviewed test sets · Fresh production data snapshots from hundreds of hospitals · False negatives · False positives · Individual hospital-level degradation · Tens of millions of scans evaluated weekly · Customer-visible accuracy changes

Outcomes

• Aidoc reports deployment in about 2,000 hospitals globally.
• Aidoc reports evaluating tens of millions of scans weekly.
• Aidoc reports a five-fold acceleration from about three algorithms per year to 15 algorithms in the past year.
• The panel establishes deterministic safeguards, context management, tracing, and evaluation infrastructure as production requirements for agents.

This is a summary-only extraction from a panel discussion. The strongest quantitative outcomes come from the Aidoc segment; other sections provide architecture guidance rather than measured deployment results.