Multi-Agent AI System for Financial Intelligence and Risk Analysis

moody’s · Finance · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Financial risk intelligence, document analysis, and customer workflow orchestration

Moody's needed production-grade generative AI for high-stakes financial intelligence where customers combine Moody's proprietary data with their own unstructured documents, ask multi-domain analytical questions, and expect accurate, cited answers for credit, climate, economics, compliance, and portfolio decisions.

Constraints:

• Customers make billion-dollar decisions from Moody's analysis, so small accuracy failures can have catastrophic business impact.
• The system must handle proprietary Moody's data, customer-uploaded PDFs, data isolation, regulatory compliance, and domain-specific financial expertise.
• Financial documents include hundreds of pages, tables, charts, footnotes, and inconsistent layouts that defeat simple text extraction or single-shot retrieval.
• Usage is spiky, so the architecture needs serverless scaling and model capacity strategies rather than fixed over-provisioning.

Architecture

Moody's evolved from a 2023 grounded research-assistant chatbot into a production multi-agent/workflow system. A custom ECS orchestrator consumes JSON workflows containing tools, agents, and prompts; parallelizes independent steps; respects dependencies; retries throttled LLM calls; and optimizes cost. About 80 Lambda-backed tools, 100+ workflows, simple prompt/tool/validation agents, and complex ECS agents use flexible per-step LLM selection through Amazon Bedrock. Unstructured financial PDFs flow through an intelligent page classifier that routes text pages to Bedrock OCR-to-markdown, table pages to Bedrock Data Automation, and charts/images to vision models plus vector metadata. Agentic retrieval plans, searches, reflects, loops, and returns cited chunks as a reusable tool for workflows and agents.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  UI["Research Assistant and Workflow Designer"] --> Orchestrator["Custom Orchestrator"]
  Orchestrator --> Tools["Lambda Tool Layer"]
  Orchestrator --> Simple["Simple Agents"]
  Orchestrator --> Complex["Complex Agents on ECS"]
  Orchestrator --> Retrieval["Agentic Retrieval Tool"]
  Tools --> Data["Moody's Data Universe"]
  Simple --> Bedrock["Bedrock Model Layer"]
  Complex --> Bedrock
  Data --> PDF["PDF Processing Pipeline"]
  PDF --> BDA["Bedrock Data Automation"]
  PDF --> Stores["Vector and Retrieval Stores"]
  BDA --> Stores
  Stores --> Retrieval
  Future["AgentCore Future Services"] --> Orchestrator
  Future --> Tools
  Future --> Complex

Components

Data flow

Users start with Moody's research assistant or visual workflow designer to orchestrate financial-intelligence tasks. The custom orchestrator receives JSON-defined workflows with tools, agents, and prompts, runs independent steps in parallel, waits for dependencies, retries throttled model calls, and uses per-step model selection through Bedrock. Lambda tools fetch data or perform calculations, while ECS-hosted complex agents handle stateful or long-running work. Customer PDFs are classified by page type, routed to Bedrock LLM OCR, Bedrock Data Automation, or vision processing, and then stored as queryable text/metadata. Agentic retrieval decomposes questions into search strategies, runs multiple searches, reflects on sufficiency, and returns cited chunks to downstream agents and workflows.

Why this architecture

Moody's chose serverless, tool-based, multi-agent workflow orchestration because financial markets create 50x traffic spikes, users build repeatable multi-step analyses rather than simple chat questions, and context switching across credit, sector, news, compliance, and customer data hurts model performance. Lambda-backed tools provide reusable stateless operations; ECS supports custom stateful and long-running agents; Bedrock gives model flexibility and regional capacity; and agentic retrieval plus page-type routing solve the retrieval limits of complex financial PDFs.

Models used: Amazon Bedrock foundation models · Claude Sonnet 4.5 via Bedrock examples · Reasoning models and small language models selected per agent/tool · Vision models for chart and image processing · Bedrock Data Automation for complex document extraction

Scale

• Moody's is described as a 100-year-old financial institution with over 1,500 customers across 165 countries.
  “We serve over 1,500 customers across 165 countries, 97% of the Fortune 100.”transcript, stated
• The platform supports 2,600 commercial banks, 1,900 asset managers, and 800+ insurance companies.
  “2,600 commercial banks processing loan originations, 1,900 asset managers making portfolio allocation decisions, and 800 plus insurance companies”transcript, stated
• Customer decisions include $500 million loan decisions and $2 billion portfolio rebalancing.
  “When a commercial bank is making $500 million loan decisions or an asset manager is rebalancing a $2 billion portfolio”transcript, stated
• Moody's Orbis database is described as containing 600 million company or entity records.
  “we also operate Orbis, which is one of the largest databases of company or an entity data in the world, this is 600 million entities.”transcript, stated
• The production agentic ecosystem has about 80 tools, 100+ workflows, many specialized agents, and over one million tokens processed per day.
  “we have about 80 or so tools. We have a hundred plus workflows, right? We have many specialized task agents themselves, and we're processing over a million tokens a day”transcript, stated
• Customer workflows grew from an expected 20-step cap to more than 400 steps.
  “our lead engineer, when he created this, he set a max limit to about 20 or so steps”transcript, stated

  “Now, some of our customer workflows are over 400 steps, right?”transcript, stated
• The research assistant produced 60% faster insights and 30% reduction in task completions.
  “It was saving people 60%, it was giving people 60% faster insights, right? It was 30% reduction in task completions.”transcript, stated

Alternatives considered (roads not taken)

Basic RAG chatbot with one model and one context window

Rejected because: It handled grounded Q&A but crumbled on complex multi-company, multi-domain analyses because a single context had shallow expertise and degraded performance.

Basic Python PDF extraction

Rejected because: It was fast and cheap but lost structural context needed for financial analysis.

Custom PDF parsing algorithms based on hierarchy and bounding boxes

Rejected because: The approach was complex and could not reliably scale across varied financial document layouts.

Multimodal foundation models over whole PDFs

Rejected because: Vision models helped but still struggled with complex tables/layouts and were too expensive at production scale.

Million-token context windows for whole documents

Rejected because: Large contexts worked better initially but degraded as documents grew and were expensive to scale.

Single-shot vector or hybrid search with top-k chunks and reranking

Rejected because: Financial questions often require scattered information across tables, pages, revenue data, sectors, and footnotes, which a single retrieval shot misses.

Patterns

multi-agent-orchestration (architecture)

Moody's runs specialized task agents and JSON-defined workflows through a custom orchestrator that coordinates tools, agents, prompts, dependencies, and parallelism.

context-engineering (architecture)

Moody's frames the architectural shift as context boundaries, domain isolation, and elimination of cross-domain interference rather than prompt tuning.

tool-use-orchestration (architecture)

The system exposes Lambda tools as reusable single-purpose operations that agents and workflows call for data fetching and calculations.

rag-retrieval (architecture)

Moody's uses Bedrock Knowledge Bases, vector stores, and agentic retrieval to retrieve cited context from proprietary and uploaded financial documents.

observability-tracing (operations)

The AgentCore direction emphasizes collecting every agent step, model call, tool call, reasoning chain, and prompt for troubleshooting and regulatory compliance.

model-routing (architecture)

Every agent, step, and tool can choose its own LLM, allowing reasoning models for complex analysis and smaller models for simpler computations.

durable-execution (operations)

The custom orchestrator manages long multi-step workflows with dependencies, retries for throttling, error handling, and ECS support for long-running agents.

safety-policy-enforcement (safety)

The system design has to account for customer data isolation, regulated financial workflows, and inbound/outbound agent authorization.

Tradeoffs

Move from better prompts to better context boundaries.

Chose Specialized agents and workflows with precise domain context over Continued prompt tweaking in a broad research assistant.

Rationale: Small prompt gains could not solve cross-domain interference and shallow expertise in high-stakes financial analysis.

Use Lambda tools and ECS complex agents together.

Chose Lambda for stateless tools and ECS for stateful or long-running agents over Using Lambda for every operation or building all agents as one service type.

Rationale: Lambda gives fast stateless reuse, while ECS handles state and tasks that exceed Lambda's runtime and compute limits.

Route PDF pages to specialized processors.

Chose Page classification with Bedrock OCR, BDA for tables, and vision models for charts/images over One universal PDF extraction method.

Rationale: Financial PDFs mix narrative text, tables, charts, images, and footnotes, and no single method met cost, accuracy, and scale requirements.

Keep custom orchestration initially but evaluate AgentCore managed primitives.

Chose Custom-built orchestration to get fast to market, with future replacement by managed Bedrock AgentCore pieces over Waiting for managed orchestration services or keeping all custom code indefinitely.

Rationale: Custom work solved missing capabilities early, but managed services may reduce technical debt now that AgentCore exists.

Allow per-agent and per-tool model selection.

Chose Flexible model routing per step over Pigeonholing the whole system into one LLM.

Rationale: Different steps need different model capabilities, costs, and validation contexts.

Gotchas (what bit them)

A single RAG assistant could not answer complex cross-domain financial questions reliably.

Root cause: The system tried to use one model and one context window across credit, sector, news, regulatory, and portfolio analysis, causing context limits and shallow expertise.

Resolution: Moody's moved to specialist intelligence, custom workflows, and context-engineered domain boundaries.

Financial PDF extraction lost meaning when structure was flattened.

Root cause: Headers, tables, footnotes, charts, and layouts can span pages, so plain text extraction removes the relationships needed for analysis.

Resolution: Moody's routes pages by modality and uses specialized processors such as Bedrock Data Automation for table-heavy pages.

Several plausible PDF strategies failed production requirements.

Root cause: Basic libraries lost structure, custom parsers did not scale, multimodal models were costly and imperfect, and large context windows degraded with document size.

Resolution: The team adopted a pipeline with page classification and modality-specific processing rather than one universal parser.

Single-shot retrieval cannot answer questions whose evidence is scattered across large annual reports.

Root cause: Business units, revenue, sector context, and footnotes may live in different tables and pages, beyond top-k search's reach.

Resolution: Moody's built agentic retrieval that plans, searches repeatedly, reflects on sufficiency, and returns cited chunks.

The orchestrator's original 20-step assumption was too small for real customer workflows.

Root cause: Production users built much more complex analyses than designers anticipated.

Resolution: The orchestration system now supports customer workflows over 400 steps.

Evaluation

Moody's reports production status, usage and workflow scale, speed improvements, task-completion reduction, grounding/citation behavior, and operational observability goals. It does not provide exact accuracy/error-rate benchmarks.

Metrics: 60% faster insights from the research assistant. · 30% reduction in task completions. · 80 or so tools, 100+ workflows, many specialized agents. · Over one million tokens processed per day. · Customer workflows over 400 steps.

Outcomes

• Moody's moved from a grounded research assistant to production multi-agent workflows serving high-stakes financial customers.
• The research assistant delivered 60% faster insights and a 30% reduction in task completions.
• The production ecosystem has about 80 tools, 100+ workflows, many specialized task agents, and more than one million tokens processed daily.
• Customer workflows grew from an expected 20-step cap to over 400 steps.
• Moody's identified managed AgentCore primitives as a way to replace custom code and reduce technical debt.

The system supports Moody's financial-intelligence customers across 165 countries, 97% of the Fortune 100, 2,600 commercial banks, 1,900 asset managers, 800+ insurance companies, and Orbis's 600 million entities.