Agentic AI Architecture for Investment Management Platform

blackrock · Finance · year n/a · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.2 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Investment management

BlackRock needed to embed an AI assistant across Aladdin applications while coordinating many domain tools and preserving responsible-AI controls.

Constraints:

100 front-end applications
Thousands of tools and agents
Access control
Responsible AI guardrails

Architecture

BlackRock's Aladdin Copilot is a supervisor-style orchestration graph for a large investment platform. A user query enters with application, portfolio, asset, widget, and global-setting context; LangChain/LangGraph input guardrails moderate toxicity and PII; filtering/access-control reduces thousands of plugin-registered tools and custom agents to a manageable candidate set; GPT-4 function calling plans and invokes tools/agents; output guardrails and hallucination checks control responses. A plugin registry lets 50-60 domain teams onboard API-backed tools or custom agents, while CI/CD evaluation tests prompt behavior and end-to-end routing daily and on every PR.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  aladdin-front-end-apps[Aladdin front-end apps] --> user-query-context[User query context]
  user-query-context --> input-guardrail-node[Input guardrail node]
  input-guardrail-node --> filtering-access-control-node[Filtering access control node]
  filtering-access-control-node --> plugin-registry[Plugin registry]
  plugin-registry --> domain-tools-and-agents[Domain tools and agents]
  filtering-access-control-node --> orchestration-node[Orchestration node]
  domain-tools-and-agents --> orchestration-node
  orchestration-node --> output-guardrails[Output guardrails]
  output-guardrails --> final-response[Final response]
  final-response --> aladdin-front-end-apps
  ci-cd-evaluation-pipeline[CI/CD evaluation pipeline] --> input-guardrail-node
  ci-cd-evaluation-pipeline --> orchestration-node
  ci-cd-evaluation-pipeline --> output-guardrails

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Aladdin front-end apps`	Application surfaces where users ask portfolio and investment-management questions.	`User query context`
`User query context`	Question plus app, screen, portfolio, asset, widget, and global-setting context.	`Input guardrail node`
`Input guardrail node`	Responsible-AI moderation, toxicity/off-topic handling, and PII identification.	`User query context`, `Filtering access control node`
`Filtering access control node`	Filters plugins by user, app, environment, and candidate relevance.	`Input guardrail node`, `Plugin registry`, `Orchestration node`
`Plugin registry`	Registry where 50-60 domain teams onboard tools and custom agents.	`Filtering access control node`, `Domain tools and agents`
`Domain tools and agents`	API-backed tools or custom agents owned by Aladdin domain teams.	`Plugin registry`, `Orchestration node`
`Orchestration node`	LangChain/LangGraph planning node using GPT-4 function calling over selected tools.	`Filtering access control node`, `Domain tools and agents`, `Output guardrails`
`Output guardrails`	Domain-specific output filtering and hallucination checks.	`Orchestration node`, `Final response`
`Final response`	Copilot response returned to the Aladdin user.	`Aladdin front-end apps`
`CI/CD evaluation pipeline`	Daily and PR-time eval reports for system prompts, routing, and intended behaviors.	`Input guardrail node`, `Orchestration node`, `Output guardrails`

Data flow

An Aladdin user asks a domain question from a front-end app. The query enters the orchestration graph with screen and portfolio context, passes input guardrails, moves through access control and plugin filtering, reaches a smaller candidate set of tools and agents, uses GPT-4 function calling to plan and call domain capabilities, applies output guardrails and hallucination checks, and is continuously evaluated through CI/CD reports.

3 supporting quotes

“So once a query gets submitted, we enter our orchestration graph. Uh from day one, it's been built using lang chain.”transcript, stated

“So once we get to that node, we go into this filtering and access control node. And at that that's the point in time where we have this, you know, set of agents and tools, couple thousands agents and tools that are registered in the plug-in registry.”transcript, stated

“So, we leave that filtering and access control node with some set of say 20 to 30 30 tools and agents and we enter our orchestration node there. We're very reliant on GPT4 uh function calling today.”transcript, stated

Why this architecture

BlackRock federated capability onboarding through a plugin registry because domain expertise lives with many Aladdin teams. Guardrails and access filtering are mandatory in finance, and reducing thousands of registered capabilities to 20-30 candidates keeps orchestration tractable. Evaluation-driven development is the confidence layer because wrong advice, routing errors, and hallucination failures can happen at multiple graph nodes.

3 supporting quotes

“I referenced this earlier, but there are say 50 60 Aladdin engineering teams that cover specific do domains throughout the platform. You can think of an an example of one of these teams maybe the trading team and they own all of the services that serve trading data to all Aladdin applications.”transcript, stated

“if we're sending more than 40 50 tools, we probably won't have uh that good of a performance. So, we leave that filtering and access control node with some set of say 20 to 30 30 tools and agents”transcript, stated

“There's many places where you can um uh make a wrong decision. It can be in the initial moderation and the guardrails, the individual components of the guardrails on the uh filtering that we not reduce the universe correctly that we did the orchestration took a wrong turn at some point.”transcript, stated

Models used: GPT-4 · LLM-as-judge

Scale

Aladdin has more than 100 front-end applications.
“And these engineers build and maintain the around 100 Aladdin front-end applications that thousands of people use every day.”transcript, stated
A couple thousand tools and agents are registered in the plugin registry.
“And at that that's the point in time where we have this, you know, set of agents and tools, couple thousands agents and tools that are registered in the plug-in registry.”transcript, stated
Evaluation reports run every day and on every PR.
“the end of the day, we have a report. This is very important part of our CI/CD pipelines. It runs every day. It runs on every PR. Why? Because we are a bunch of developers. Every day we are improving the system. Every day we are releasing to our development environment and you want to know”transcript, stated

Alternatives considered (roads not taken)

Central team builds every domain capability

Rejected because: The Copilot team is not expert in every financial domain, so domain teams own tools and agents.

“Pedro and I are not uh experts in these uh you know business financial domains. Our job is to make it as as easy as possible for these engineers to plug their functionality that they own into this system and we do that through this plug-in registry.”transcript, stated

Send all registered tools to the planner

Rejected because: Too many tools hurt performance, so access control and filtering reduce the candidate universe.

“So, we leave that filtering and access control node with some set of say 20 to 30 30 tools and agents and we enter our orchestration node there.”transcript, implied

Rely on system prompts without behavior evals

Rejected because: Financial constraints such as no investment advice require synthetic and SME data in CI/CD eval pipelines.

“So we toss um start by testing every single behavior that we uh are intending by writing that in the system prompt. So for example, if I write you're a lot in copilot, you must never provide investment advice.”transcript, stated

Patterns

`multi-agent-orchestration` (platform)

A supervisor graph routes user requests across tools and agents owned by many engineering teams.

“user. So once a query gets submitted, we enter our orchestration graph. Uh from day one, it's been built using lang”transcript, stated

`safety-policy-enforcement` (governance)

The architecture includes guardrail and access-control nodes before tool execution.

“chain. So the first node in that graph uh is an input guardrail node um and and it covers a lot of the the needs for our”transcript, stated

Tradeoffs

Use statistical evaluation to govern federated agent development.

Chose Daily end-to-end evals showing whether the system is improving or deteriorating over Debugging enterprise rollout through isolated complaints about individual failed queries.

Rationale: Federated teams need aggregate evidence that routing and answer quality are stable enough to scale.

“you need a statistical way of saying that your system is working is improving it's not deteriorating so that you can continue to build the scalable products otherwise it's it's very hard”transcript, stated

How to scale domain capability onboarding.

Chose federated plugin registry for tools and agents over centralized Copilot team implementation.

Rationale: Domain teams own specialized APIs and workflows, while the Copilot team provides the registry and orchestration substrate.

“make it as as easy as possible for these engineers to plug their functionality that they own into this system”transcript, stated

How many tools to expose to planning.

Chose filter to roughly 20-30 candidate tools and agents over passing thousands of registered capabilities into the planner.

Rationale: The system reduces the searchable universe before GPT-4 function calling to preserve routing quality.

“with some set of say 20 to 30 30 tools and agents”transcript, stated

Gotchas (what bit them)

Financial copilots need explicit behavior tests for prohibited advice.

Root cause: A system prompt such as never provide investment advice is not enough unless the intended behavior is exercised with domain examples.

Resolution: BlackRock generated subject-matter-expert data and built evaluation pipelines for each intended prompt behavior.

“we are very paranoid about telling the user the wrong things. So we toss um start by testing every single behavior that we uh are intending by writing that in the system prompt.”transcript, stated

Evaluation

BlackRock used automated CI/CD evaluation of prompts and routing behavior.

Metrics: prompt behavior tests · query routing tests

“Blackrock implemented rigorous evaluation-driven development with automated testing pipelines running in CI/CD, testing every system prompt behavior and end-to-end query routing to ensure reliable performance across thousands of registered tools and agents.”summary, stated

Outcomes

The assistant is embedded across a large Aladdin application surface with many onboarded teams.

The summary reports 50-60 teams onboarding tools and agents.

“The solution employs a supervisor-based agentic architecture built on LangChain/LangGraph, featuring a plugin registry that allows 50-60 engineering teams to onboard their domain-specific tools and agents.”summary, stated