Production AI Agents for Insurance Policy Management with Amazon Bedrock

cdl · Insurance · year n/a · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Insurance policy administration and customer self-service

CDL needed production AI agents that could let authenticated insurance customers manage policies conversationally while safely invoking existing policy APIs and respecting regulated insurance constraints.

Constraints:

• Insurance answers must be complete and conditioned rather than simple yes/no responses.
• Existing policy-administration APIs were built for programmatic clients, not LLM agents.
• High-risk topics such as credit history, crime, claims, and profanity require blocking or human escalation.
• Model choice changes quickly, so CDL needs repeatable model evaluation rather than a one-time selection.

Architecture

CDL uses Amazon Bedrock as an AI front door for insurance policy-management journeys. Authenticated customer intent is routed to a Bedrock supervisor agent, which delegates to domain-specific helper agents. Each domain agent receives OpenAPI tool specifications and customer policy context at runtime, then calls Lambda functions in an AI integration/anti-corruption layer that maps agent actions to existing CDL policy APIs. Bedrock evaluations, LLM-as-judge scoring, human expert review, monitoring hooks, and Bedrock Guardrails surround the flow.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  customer-insurance-portal["Customer insurance portal"] --> ai-front-door["AI front door"]
  ai-front-door["AI front door"] --> bedrock-supervisor-agent["Bedrock supervisor agent"]
  bedrock-supervisor-agent["Bedrock supervisor agent"] --> domain-policy-agents["Domain policy agents"]
  domain-policy-agents["Domain policy agents"] --> openapi-action-groups["OpenAPI action groups"]
  domain-policy-agents["Domain policy agents"] --> ai-integration-layer["AI integration layer"]
  ai-integration-layer["AI integration layer"] --> existing-cdl-apis["Existing CDL APIs"]
  ai-integration-layer["AI integration layer"] --> monitoring-hooks["Monitoring hooks"]
  existing-cdl-apis["Existing CDL APIs"] --> policy-administration-system["Policy administration system"]
  bedrock-evaluations["Bedrock evaluations"] --> domain-policy-agents["Domain policy agents"]
  bedrock-evaluations["Bedrock evaluations"] --> human-domain-experts["Human domain experts"]
  bedrock-guardrails["Bedrock Guardrails"] --> ai-front-door["AI front door"]
  bedrock-guardrails["Bedrock Guardrails"] --> domain-policy-agents["Domain policy agents"]
  bedrock-guardrails["Bedrock Guardrails"] --> specialized-human-agent["Specialized human agent"]

Components

Data flow

A customer authenticates in the insurance portal and expresses an intent. The front end injects identity and policy context into the prompt and sends the intent to a Bedrock supervisor agent. The supervisor delegates to a domain agent, whose OpenAPI action group tells the LLM which tool path and schema are available. When the LLM has the needed policy reference and parameters, it calls a Lambda integration layer, which maps the request onto existing policy-administration APIs. Guardrails can intercept risky input before the model or risky output after generation.

Why this architecture

The supervisor/domain-agent shape lets CDL map agent responsibilities to insurance business domains while preserving existing policy APIs behind an AI-specific integration layer. OpenAPI action groups constrain tool use to known schemas, and Bedrock evaluations/guardrails give CDL a repeatable way to compare models and block regulated topics before broad release.

Models used: Amazon Bedrock-hosted LLMs · Anthropic models · Amazon models · DeepSeek models · Meta models · Mistral models · LLM-as-judge evaluator model

Scale

• The guardrail setup can be assigned to multiple AI agents after creation.
  “this guardrail is now in place and I can use that in multiple different like AI architectures. So I might have like two AI agents.”transcript, stated
• A demo evaluation used 25 insurance questions against ground-truth answers.
  “I've uploaded 25 insurance questions to this before the show and I've said evaluate the answers that the LLM gives against my ground truth data”transcript, stated
• CDL compared a wide Bedrock model set, including Amazon, Anthropic, DeepSeek, Meta, and Mistral options.
  “I've enabled all of the Amazon ones, all the anthropic ones, all the uh Deep Seek ones in there, Meta one, uh Mistral”transcript, stated

Alternatives considered (roads not taken)

Let agents call existing APIs directly without an AI integration layer

Rejected because: CDL says those APIs were designed for programmatic clients, so an AI-specific Lambda anti-corruption layer makes agent interaction safer and easier.

Choose a single model without ongoing comparison

Rejected because: CDL treats model choice as use-case-specific and runs Bedrock evaluations across many enabled models.

Rely only on automated model-as-judge evaluation

Rejected because: The automated pass is backed by human expert evaluation from legal, compliance, and domain specialists.

Patterns

multi-agent-orchestration (architecture)

A Bedrock supervisor agent routes work to specialized helper agents aligned with insurance business domains.

tool-use-orchestration (architecture)

Domain agents receive OpenAPI tool specs at runtime, including paths and schemas required to call policy-changing actions.

structured-output-contract (tooling)

Tool schemas define required fields such as policy reference and boolean consent before the LLM can call an API.

eval-driven-development (evaluation)

CDL runs Bedrock evaluations whenever models, prompts, or configurations change, using domain-grounded questions and metrics.

llm-as-judge (evaluation)

Automated evaluations use one LLM to answer and another to grade against ground truth across completeness and hallucination-related factors.

human-in-the-loop-gate (governance)

Human domain experts, legal, and compliance teams define and assess evaluation packs for regulated insurance responses.

safety-policy-enforcement (safety)

Bedrock Guardrails block risky insurance topics before model invocation and can also inspect model outputs.

observability-tracing (operations)

The Lambda integration includes monitoring hooks around tool execution, shown through Dynatrace instrumentation.

Tradeoffs

Agent boundary design

Chose Supervisor plus domain-specific helper agents over One broad agent for all insurance policy operations.

Rationale: Domain-driven design maps agents to business domains, giving CDL clearer boundaries for policy-management tasks.

Integration strategy

Chose AI integration layer of Lambda functions over Letting Bedrock agents directly use legacy policy APIs.

Rationale: The existing APIs were not designed for AI interaction, so Lambda mapping reduces coupling and adapts them for agent use.

Evaluation strategy

Chose LLM-as-judge backed by human expert evaluation over Automated LLM scoring alone.

Rationale: Insurance correctness depends on regulated domain nuance, so automated comparisons need human legal/compliance/domain oversight.

Risk handling

Chose Guardrail block and human escalation over Allowing the LLM to answer sensitive high-risk questions.

Rationale: CDL prefers to redirect risky topics such as crime or claims to specialized human agents.

Gotchas (what bit them)

Existing APIs were not AI-ready.

Root cause: Policy APIs were designed for deterministic programmatic clients rather than LLM-driven interactions.

Resolution: CDL inserted an AI integration/anti-corruption layer of Lambda functions between agents and existing APIs.

Insurance answers can be technically correct but incomplete.

Root cause: Policy questions often require conditions and caveats rather than binary answers.

Resolution: Evaluation metrics explicitly include completeness, correctness, faithfulness to ground truth, and domain-expert review.

High-risk policy topics need escalation rather than generative free-form answers.

Root cause: Insurance conversations may involve credit history, criminal records, claims, or other sensitive risk topics.

Resolution: CDL configures Bedrock Guardrails to block or redirect those topics to specialized human agents.

Runtime context must include both authenticated customer state and tool schemas.

Root cause: The model cannot safely execute policy actions without policy reference, user authentication, and the relevant action schema.

Resolution: The front end injects authenticated policy context while Bedrock injects OpenAPI tool specs into the prompt.

Evaluation

CDL uses Bedrock evaluations with curated insurance questions, domain-expert ground truth, automated LLM-as-judge scoring, and human review through SageMaker/Mechanical Turk-backed workflows.

Metrics: completeness · correctness · faithfulness to ground truth · hallucination risk · responsible AI factors · human expert assessment

Outcomes

• CDL demonstrated an authenticated policy-management flow that can set auto-renewal consent through Bedrock agent tool use.
• Guardrails blocked a high-risk criminal-record insurance question before it reached the model.
• The evaluation process lets CDL compare many Bedrock-hosted models against the same insurance-specific ground truth.

The transcript is a production-oriented demonstration rather than a numeric business outcome report. The clearest scale signals are the multiple Bedrock model families, 25-question demo evaluation, and reusable guardrail IDs assignable to multiple agents.