Building a Microservices-Based Multi-Agent Platform for Financial Advisors

prudential · Insurance · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Financial-services advisor platform

Prudential needed a scalable agentic platform for life-insurance advisors who were spending too much time moving across fragmented carrier systems, while enterprise teams also needed reusable governance, security, observability, and deployment layers for many future AI agents.

Constraints:

Architecture

Prudential and AWS built a microservices-based multi-agent platform around a life-insurance advisor assistant. Advisors use a chat UI that authenticates through SSO, passes a secure token and context window ID into an orchestration agent, and routes intent to specialized sub-agents for quick quotes, forms, product information, illustrations, and book-of-business actions. Agents use an LLM gateway, knowledge management, guardrails, MCP/API access, memory, A2A transitions, monitoring, evaluation, and enterprise DevSecOps services.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  advisor-chat-ui["Advisor chat UI"] --> sso-authentication["SSO authentication"]
  sso-authentication --> agentic-layer["Agentic layer"]
  agentic-layer --> orchestration-agent["Orchestration agent"]
  orchestration-agent --> quick-quote-agent["Quick quote agent"]
  orchestration-agent --> forms-agent["Forms agent"]
  orchestration-agent --> product-agent["Product agent"]
  orchestration-agent --> illustration-agent["Illustration agent"]
  orchestration-agent --> book-of-business-agent["Book of business agent"]
  orchestration-agent --> guardrails["Guardrails"]
  orchestration-agent --> a2a-protocol["A2A protocol"]
  quick-quote-agent --> llm-gateway["LLM gateway"]
  forms-agent --> llm-gateway
  product-agent --> llm-gateway
  quick-quote-agent --> knowledge-management-system["Knowledge management system"]
  forms-agent --> knowledge-management-system
  product-agent --> knowledge-management-system
  book-of-business-agent --> knowledge-management-system
  quick-quote-agent --> training-and-validation-pipelines["Training and validation pipelines"]
  a2a-protocol --> external-multi-agent-systems["External multi-agent systems"]
  agent-core-platform["Agent core platform"] --> mcp-tool-access["MCP tool access"]
  agent-core-platform --> llm-gateway
  agent-core-platform --> monitoring-and-evaluation["Monitoring and evaluation"]
  agent-core-platform --> sagemaker-inference["SageMaker inference"]
  agent-core-platform --> enterprise-infrastructure["Enterprise infrastructure"]
  mcp-tool-access --> sagemaker-inference
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Advisor chat UI Natural-language interface where advisors ask questions and request support across life-insurance workflows. SSO authentication, Orchestration agent
SSO authentication Authenticates users based on access controls and issues a secure token. Advisor chat UI, Agentic layer
Agentic layer Revalidates secure tokens, creates context window IDs, and passes session context into the agent system. SSO authentication, Orchestration agent
Orchestration agent Single entry point that interprets advisor intent, maintains context, and routes requests to specialized sub-agents. Advisor chat UI, Agentic layer, Quick quote agent, Forms agent, Product agent, Illustration agent, Book of business agent, Guardrails, A2A protocol
Quick quote agent Underwriting-like LLM application that asks follow-up health questions and returns risk-rating guidance. Orchestration agent, LLM gateway, Knowledge management system, Training and validation pipelines
Forms agent Retrieves the right insurance forms and asks follow-up questions about transaction type and state. Orchestration agent, LLM gateway, Knowledge management system
Product agent Smart-search agent for product questions, features, suitability, and product-related advisor requests. Orchestration agent, LLM gateway, Knowledge management system
Illustration agent Serves product illustrations through API-like access inside the agent framework. Orchestration agent, MCP tool access
Book of business agent Helps advisors act on placed policies and next-best-action recommendations across their book of business. Orchestration agent, MCP tool access, Knowledge management system
LLM gateway Central secure gateway for sub-agents to access models. Quick quote agent, Forms agent, Product agent, Knowledge management system
Knowledge management system Document and vector-store layer used by agents for retrieval and response generation. LLM gateway, Quick quote agent, Forms agent, Product agent, Book of business agent
Guardrails Functional guardrails at both orchestration and sub-agent levels for invalid queries, subqueries, and follow-up handling. Orchestration agent, Quick quote agent, Forms agent, Product agent
MCP tool access Standardized tool and resource access for individual agents and traditional model services. Orchestration agent, SageMaker inference, Agent core platform
A2A protocol Agent-to-agent transition layer for moving between agents and other multi-agent systems. Orchestration agent, External multi-agent systems
Monitoring and evaluation Agent span, traceability, observability, and evaluation framework under platform management. Agent core platform, Orchestration agent
Agent core platform Reusable enterprise platform layer for Bedrock access, context engineering, gateways, registries, DevSecOps, data, and infrastructure. MCP tool access, A2A protocol, LLM gateway, Monitoring and evaluation, SageMaker inference, Enterprise infrastructure
SageMaker inference Traditional model-serving layer that agents can access through MCP. MCP tool access, Agent core platform
External multi-agent systems Other Prudential systems such as retirement and intelligent-document-processing multi-agent systems reached through A2A. A2A protocol
Training and validation pipelines Pipelines for prompt optimization, validation, and follow-up-question selection in the quick-quote flow. Quick quote agent
Enterprise infrastructure Enterprise services such as GitHub Actions, DevSecOps, Splunk, ServiceNow, AWS infrastructure, and data services. Agent core platform

Data flow

An advisor signs into the UI through SSO. The system generates a secure token, revalidates it in the agent layer, maps it to a context-specific window ID, and passes that pair to the orchestration agent. The orchestrator interprets the advisor's intent, maintains session context, routes to a specialized sub-agent, and that agent may call the LLM gateway, knowledge management system, MCP tools, APIs, memory, or another agent before returning the response through the assistant.

4 supporting quotes
“user interacts with a UI APP, where an SSO authentication happens, and the SSO authentication happens based on the user's home access level control.”transcript, stated
“Once the token is validated, it generates a context specific Window ID, and maps it back to the secure token. These two pairing is again passed to the orchestration agent, and orchestration agent builds that entire thing,”transcript, stated
“we have an orchestration agent, which is a single point of entry for advisors. It takes different sorts of inputs from the advisors, it understands intent and it is non-deterministic in nature.”transcript, stated
“they use LLM gateway, which is another secure way of accessing LLM, in order to get response, and if an agent wants to use knowledge management, it uses the knowledge management system to get the relevant documents”transcript, stated

Why this architecture

Prudential moved from entangled agent solutions toward modular microservices and a shared platform because enterprise AI needed independent ownership, secure governance, reusable common layers, rapid model/prompt/framework upgrades, and cross-business reuse. The architecture separates advisor-facing business logic from authentication, context management, LLM access, knowledge management, monitoring, evaluation, MCP/A2A access, and enterprise infrastructure.

4 supporting quotes
“Agent implementations are entangled and some components are shared. So the boundaries of ownership gets blurred.”transcript, stated
“It allows you to separate concerns and easily swap components when necessary. And identifying key components like runtime, memory, code interpreter and browser tool will go a long way, and this will allow you to adapt”transcript, stated
“but platform-based services allows us to do that particular, and share the common layer piece, and from a agentic monitoring and evolution point, standpoint also, we are evaluating and deploying something which basically helps us to”transcript, stated
“These modular solutions will actually help people on the top layer in deploying things at a scale. They can reuse the agent monitoring, or agent development deployment, like, any of those frameworks on the top, from the bottom, and deploy”transcript, stated

Models used: AWS Bedrock · SageMaker inference · LLMs behind LLM gateway

Scale

Alternatives considered (roads not taken)

Put many agentic solutions in a single repository

Rejected because: Ownership boundaries blurred, deployments slowed, and sensitive-data handling became difficult.

“they build multiple solutions, multiple agentic solutions within one repo, which might be the same repo that you might have developed two to three years ago for the RAG application, right?”transcript, stated
“But this starts to break down when there are too many agents and there are such things as, too many of something, right? Agent implementations are entangled and some components are shared.”transcript, stated

Deploy each multi-agent system bespoke

Rejected because: Bespoke deployments rebuild common layers instead of sharing platform capabilities.

“if you develop a agentic, multi-agentic system, right, and deploy it bespoke, you will not able to, able to share the common layers”transcript, stated

Treat every business problem as an agent problem

Rejected because: Some use cases, including quick quote, are better understood as standalone LLM applications or need workflow/rule systems around them.

“your agents are not suitable for all business problems. Some problems like, for quick code that I mentioned, it's not an agentic solution by itself, it is more like a”transcript, stated
“it's not an agentic solution by itself, it is more like a standalone LLM application, where for a particular task, you are trying to replicate an underwriter using a complex LLM application system by design.”transcript, stated

Recreate memory and observability for every multi-agent system

Rejected because: Per-use-case databases, caches, memory, and context engineering do not scale across changing business systems.

“for every multi-agent system, it's hard to actually recreate its own database, its own caching memory, its own like, engineer, like, short term, long term memory, or like static memories, and building context”transcript, stated
“use case by use case, it becomes very harder to scale. And things like agent core, which actually provide some of these features inbuilt, which most of the industry”transcript, stated

Custom-build generic platform capabilities instead of adopting emerging industry foundations

Rejected because: For generic use cases, the team prefers industry-standard foundations such as Agent Core and reserves custom engineering for complex edge cases.

“for all the generic purpose use cases, it's probably better to stick with something industry is heading towards.”transcript, stated
“If you have some edge cases, or use cases which are very complicated, then probably, you can probably try to re-engineer that.”transcript, stated

Patterns

multi-agent-orchestration (agents)

A central orchestration agent interprets advisor intent and routes each request to specialized quick-quote, forms, product, illustration, or book-of-business agents.

“we have an orchestration agent, which is a single point of entry for advisors.”transcript, stated
“So it routes to multiple subagents, in terms of what the context is, based on the context that advisor's asking”transcript, stated

tool-use-orchestration (agents)

Agents access MCP tools, APIs, SageMaker-served models, knowledge management, and other systems through platform-mediated interfaces.

“we are actively developing and adding more agents, and adding support for MCP-based tool access, as well as API access for the agentic layer.”transcript, stated
“traditional model space is being catered by the SageMaker inference which is, which is also sometime, agent uses it via MCP.”transcript, stated

rag-retrieval (retrieval)

Agents that need documents call a central knowledge management system to retrieve relevant material before generating responses.

“if an agent wants to use knowledge management, it uses the knowledge management system to get the relevant documents and generate its response”transcript, stated
“agentic AI, which has a vector store available to it, in for knowledge management system.”transcript, stated

guardrails-io-filtering (safety)

Guardrails exist both at the orchestration layer and inside individual agents to handle invalid queries and follow-up interactions.

“we have guardrails established, from a functional standpoint, we have guard rails established at each agent and also at orchestration agent”transcript, stated
“that actually handles like, your invalid queries or subqueries, or also handles some kind of like, follow ups, back and forth with this.”transcript, stated

agent-memory (agents)

The platform treats short-term memory, long-term memory, static memory, and context-window IDs as shared infrastructure problems rather than per-agent afterthoughts.

“it uses the memory management for short term memory, as well as long term memory access”transcript, stated
“for every multi-agent system, it's hard to actually recreate its own database, its own caching memory, its own like, engineer, like, short term, long term memory”transcript, stated

observability-tracing (operations)

Prudential is standardizing agent monitoring, spans, traceability, debugging, prompt-chain observability, and production monitoring through the platform.

“we are evaluating and deploying something which basically helps us to measure the entire agent span, as well as agent traceability, as well as observability into it.”transcript, stated
“understanding the chain of prompts from an observability standpoint, monitoring a solution once it goes into production”transcript, stated

safety-policy-enforcement (safety)

Centralized platform services enforce access control, responsible AI, governance, and compliance across agents instead of leaving each solution to implement them separately.

“Enterprise need to manage the access control of their users and agents for their resources”transcript, stated
“since we have a platform based approach, right, we were able to ensure that responsible AI is engaged at that level.”transcript, stated

Tradeoffs

Enterprise agent architecture

Chose microservices and shared platform layers over many entangled agent solutions in one repository.

Rationale: Modularity preserves ownership boundaries, allows independent scaling, and lets teams swap fast-changing GenAI components.

“we want to take some components out and separate the concerns for business logic, runtime execution, governance and scaling.”transcript, stated
“modularity is a key. It allows you to separate concerns and easily swap components when necessary.”transcript, stated

Advisor assistant topology

Chose orchestration agent plus specialized sub-agents over one monolithic advisor bot.

Rationale: Specialized agents can own quick quotes, forms, product search, illustrations, and book-of-business workflows while the orchestrator maintains context and routing.

“So it routes to multiple subagents, in terms of what the context is, based on the context that advisor's asking”transcript, stated
“each agent is a specialized entity that is trained in the process to actually deliver, with more accurate responses”transcript, stated

LLM access and governance

Chose central LLM gateway and responsible-AI platform controls over direct model calls by each sub-agent.

Rationale: A centralized gateway and platform layer give the enterprise control over access, governance, monitoring, and compliance.

“they use LLM gateway, which is another secure way of accessing LLM, in order to get response”transcript, stated
“since we have a platform based approach, right, we were able to ensure that responsible AI is engaged at that level.”transcript, stated

Workflow automation boundary

Chose end-to-end business processes with rules and workflow management when needed over generic agents for every task.

Rationale: Some insurance workflows only deliver value when IDP, business rules, and workflow management are combined.

“your agents are not suitable for all business problems.”transcript, stated
“from an end-to-end value standpoint, you probably need other components on top of your IDP itself.”transcript, stated

Platform evolution strategy

Chose adopt industry-standard agent foundations for generic use cases over re-engineering every generic platform capability internally.

Rationale: The team wants to focus custom engineering on genuinely complex edge cases while tracking where the industry is heading.

“if they're actually doing all of the groundwork, maybe it's better to just adopt this as part of your platform”transcript, stated
“If you have some edge cases, or use cases which are very complicated, then probably, you can probably try to re-engineer that.”transcript, stated

Gotchas (what bit them)

Putting many agents into one repository broke down as the number of agents grew.

Root cause: Shared components blurred ownership, security alerts crossed team boundaries, and sensitive-data logic became solution-specific.

Resolution: Prudential separated business logic, runtime execution, governance, and scaling into modular platform services.

“Agent implementations are entangled and some components are shared. So the boundaries of ownership gets blurred.”transcript, stated
“making one line of change might trigger security alerts from multiple teams.”transcript, stated

Context engineering became difficult at 100,000-user scale.

Root cause: Each session pairs secure tokens, context window IDs, orchestration context, model behavior, and sub-agent interactions.

Resolution: The team is developing context-engineering frameworks, observability, spans, and traceability to debug performance drops.

“the context engineering around this hundred thousand user base, which is paired with a secure token, and the context Window ID is, becomes very challenging”transcript, stated
“debugging this context engineering frame, in order to span, separate it out and find out exactly why it happened. It is challenging in this environment.”transcript, stated

Agents are not suitable for every business problem.

Root cause: Some workflows are task-specific LLM applications or require IDP, rules, and workflow management rather than a generic agent.

Resolution: The team explicitly distinguishes standalone LLM applications from truly agentic systems and designs end-to-end value chains.

“your agents are not suitable for all business problems.”transcript, stated
“you need IDP, plus you need a business rule processing system on top of it, or you need like, some kind of workflow management”transcript, stated

Production agent performance can drop unpredictably.

Root cause: Provider model upgrades and incomplete training/validation coverage change behavior after deployment.

Resolution: The platform adds standardized evaluation, monitoring, tracing, and context debugging to detect and diagnose drops.

“there's an unpredictable drop in agent performance, and it happens a lot when you try to scale in production.”transcript, stated
“one, there's model upgrades that are happening, which probably are not, or probably affecting the way the agents actually perform.”transcript, stated

Per-use-case memory and observability architectures do not scale across enterprise agent systems.

Root cause: Each new system would otherwise recreate databases, caches, short-term memory, long-term memory, static memory, and context engineering.

Resolution: Prudential is standardizing memory, observability, and context engineering through platform capabilities and industry frameworks.

“for every multi-agent system, it's hard to actually recreate its own database, its own caching memory”transcript, stated
“having a standardized approach on memory and observability, and also like, context engineering, might help in scaling in longer run”transcript, stated

Evaluation

Prudential uses training and validation pipelines for prompt optimization in the quick-quote system and is adding platform-level monitoring, observability, agent spans, traceability, and evaluation frameworks to assess production behavior.

Metrics: training and validation pipeline results · prompt optimization validation · agent spans · agent traceability · observability signals · business feedback · turnaround time

3 supporting quotes
“we have built training of validation pipelines that will automatically figure out prompt optimization validation”transcript, stated
“we are evaluating and deploying something which basically helps us to measure the entire agent span, as well as agent traceability”transcript, stated
“understanding the chain of prompts from an observability standpoint, monitoring a solution once it goes into production”transcript, stated

Outcomes

The system is deployed for more than one hundred thousand advisors, supports advisor workflows that previously touched 50 or 60 IT systems, and targets enterprise reuse across retirement, group insurance, life insurance, underwriting, claims, marketing, and other business functions.

4 supporting quotes
“We are live today with more than a hundred thousand advisors actually using this”transcript, stated
“reduce the turnaround time, from like six to eight weeks earlier, to like, three to four weeks, now.”transcript, stated
“we are able to add multiple agents on top top of this, and able to deploy into production in like, four to five weeks time”transcript, stated
“we are now seeing value of, how do we can reuse this particular development across, in the same way in other business unit.”transcript, stated