Multi-Agent GenAI System for Developer Support and Documentation
northwestern_mutual · Insurance · 2023 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)
Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.
Problem
Domain: Internal developer support
Northwestern Mutual needed faster internal developer support for repetitive documentation questions and automatable account tasks in a regulated financial-services environment.
Constraints:
- Support response times were long during weekends and holidays.
- Many questions already had documentation but still consumed engineer time.
- The first production use case had to establish patterns suitable for more regulated business-facing applications.
- The solution needed specialized handling for documentation, accounts, repositories, pipelines, and response quality.
Architecture
Northwestern Mutual built an Amazon Bedrock multi-agent support bot that moved from experiment to production in about three months. Specialized agents handle documentation, user management, repository management, pipeline-failure analysis, and response evaluation, allowing routine developer-support requests to be answered or acted on without direct engineer engagement.
Diagram confidence: reconstructed (reconstructed from prose).
flowchart LR support-bot["Support Bot"] documentation-agent["Documentation Agent"] user-management-agent["User Management Agent"] repository-agent["Repository Agent"] pipeline-failure-agent["Pipeline Failure Agent"] response-evaluation-agent["Response Evaluation Agent"] support-bot --> documentation-agent support-bot --> user-management-agent support-bot --> repository-agent support-bot --> pipeline-failure-agent support-bot --> response-evaluation-agent documentation-agent --> support-bot user-management-agent --> support-bot repository-agent --> support-bot pipeline-failure-agent --> support-bot response-evaluation-agent --> support-bot
Components
| Component | Role | Interacts with |
|---|---|---|
Support Bot |
Receives internal developer-support requests and coordinates specialized agents. | Documentation Agent, User Management Agent, Repository Agent, Pipeline Failure Agent, Response Evaluation Agent |
Documentation Agent |
Answers repetitive questions using existing documentation. | Support Bot |
User Management Agent |
Handles automatable account tasks such as unlocking user accounts. | Support Bot |
Repository Agent |
Supports repository-management questions and actions. | Support Bot |
Pipeline Failure Agent |
Analyzes build or pipeline failures for developers. | Support Bot |
Response Evaluation Agent |
Evaluates support responses before completion. | Support Bot |
Data flow
A developer asks for help through the internal support channel. The support bot routes the request to a documentation, user-management, repository, pipeline-analysis, or response-evaluation agent, then returns an answer or executes an automatable support task while preserving the option for complex issues to remain with engineers.
Why this architecture
The support workflow contains several distinct task families, so Northwestern Mutual split the bot into specialized agents rather than asking one agent to cover documentation, user management, repositories, pipeline failures, and answer evaluation.
Models used: Amazon Bedrock models
Scale
- The production path took about three months from June to September 2024.“moving from experimentation to production in approximately three months (June to September 2024).”summary, stated
- The bot answers support requests within minutes without engineer engagement.“Key results include answering support requests within minutes without engineer engagement”summary, stated
Alternatives considered (roads not taken)
Keep developer support fully engineer-handled
Rejected because: Long response times and repetitive basic questions pulled engineers away from complex issues.
Answer only documentation questions
Rejected because: The support queue also included automatable tasks such as unlocking user accounts.
Patterns
multi-agent-orchestration (architecture)
The bot delegates to documentation, user-management, repository, pipeline, and evaluation agents.
tool-use-orchestration (execution)
The support bot handles automatable internal tasks in addition to answering questions.
Tradeoffs
Start with internal developer support
Chose A regulated but internal production use case over Beginning with more complex business-facing applications.
Rationale: The internal support bot established a foundation for later business-facing applications.
Specialize agents by support domain
Chose Separate documentation, user, repo, pipeline, and evaluation agents over A single broad support agent.
Rationale: The request types were distinct enough to justify specialist behavior and evaluation.
Gotchas (what bit them)
Documentation alone did not prevent repetitive support requests.
Root cause: Developers still asked basic questions even when answers already existed.
Resolution: The bot routed documentation questions to a specialized agent.
Support coverage gaps were worse outside normal engineer availability.
Root cause: Weekends and holidays made human support slower.
Resolution: The Bedrock support bot could answer routine requests within minutes without engineer engagement.
Evaluation
The architecture includes a response evaluation agent, and outcomes are reported as response-time and engineer-focus improvements.
Metrics: minutes to answer support requests · approximately three months from experiment to production
Outcomes
- Routine developer-support requests could be answered within minutes.
- Support engineers were freed to focus on complex issues.
- The project created a production foundation for later regulated GenAI applications.
The reported timeline was June to September 2024, with support answers within minutes.