Building Multi-Agent AI Systems for Developer Support and Infrastructure Operations

electrolux · Tech · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Internal developer-support and infrastructure-operations agents

Electrolux's platform/SRE team became the support interface for cloud infrastructure, access management, internal developer platform questions, and third-party tooling. Support arrived through Slack, and the small SRE team could become a bottleneck during busy periods. The team built Infra Assistant to answer organization-specific questions, take operational actions, troubleshoot cloud issues, and augment capacity without forcing developers to wait for a human SRE for repetitive tasks.

Constraints:

• Answers had to use Electrolux-specific documentation, terminology, and prior conversations rather than generic cloud knowledge.
• The assistant needed to perform actions, not only chat, while avoiding unsafe reliance on the LLM for sensitive operations.
• The system had to keep a helpful internal-tool tone and earn developer trust with references to ground truth.
• Agent complexity, tool confusion, prompt size, slowness, and cost grow as a single agent accumulates tools and data.
• Stateful deployment of a large agent tree slowed prompt and tool iteration.
• Troubleshooting cloud infrastructure required access to logs, code, knowledge bases, and AWS account state under safe permissions.
• The system was not 100% accurate, hallucinations occurred, and complex tasks could take up to several minutes.

Architecture

Infra Assistant started as a Bedrock knowledge-based RAG agent exposed through a service interface for developers. It then evolved into a multi-agent system with a main supervisor that reasons over specialized agents: knowledge-based agents, API agents backed by OpenAPI specifications and Lambda calls, classic action-group agents with optional user confirmation, onboarding/Atlassian agents, MCP-derived action groups, and a cloud-troubleshooting agent. The deployment model moved from code-defined agent trees deployed through CI/CD into Bedrock to inline agents: the Infra Assistant service dynamically generates the whole agent tree as JSON alongside each question, Bedrock compiles it with minimal delay, and the supervisor delegates work to sub-agents. Bedrock tracing, developer feedback, SRE-channel ratings, read-only IAM roles, and source references provide observability, evaluation, and safety boundaries.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  slack-support-channel["Slack support channel"] --> infra-assistant-service["Infra Assistant service"]
  infra-assistant-service --> bedrock-inline-agents-runtime["Bedrock inline agents runtime"]
  bedrock-inline-agents-runtime --> main-supervisor-agent["Main supervisor agent"]
  main-supervisor-agent --> knowledge-based-supervisor["Knowledge-based supervisor"]
  knowledge-based-supervisor --> knowledge-based-agents["Knowledge-based agents"]
  knowledge-based-agents --> knowledge-base["Knowledge base"]
  main-supervisor-agent --> api-agents["API agents"]
  api-agents --> lambda-action-groups["Lambda action groups"]
  lambda-action-groups --> user-confirmation-gate["User confirmation gate"]
  main-supervisor-agent --> mcp-server-adapters["MCP server adapters"]
  main-supervisor-agent --> troubleshooting-supervisor["Troubleshooting supervisor"]
  troubleshooting-supervisor --> log-management-system["Log management system"]
  troubleshooting-supervisor --> code-search["Code search"]
  troubleshooting-supervisor --> aws-cli-agent["AWS CLI agent"]
  aws-cli-agent --> read-only-iam-role["Read-only IAM role"]
  bedrock-inline-agents-runtime --> bedrock-tracing["Bedrock tracing"]
  infra-assistant-service --> feedback-ratings["Feedback ratings"]

Components

Data flow

A developer asks Infra Assistant in the shared SRE Slack channel or privately. The Infra Assistant service sends Bedrock an inline JSON agent tree and the user's question. The main supervisor reasons about the request, chooses specialized knowledge, API, action-group, MCP, onboarding, or troubleshooting agents, and delegates subtasks. Knowledge agents retrieve organization-specific documentation and previous chats; API agents parse OpenAPI specs and call Lambda-backed integrations; action groups can request user confirmation; the troubleshooting supervisor can query logs, search code, consult the knowledge base, and run AWS CLI commands in a restricted read-only environment. Bedrock traces expose the plan, delegated steps, selected APIs, and final answer; developers can rate the interaction and add feedback.

Why this architecture

The team chose multi-agent collaboration because a single agent becomes complex, confused, slow, and expensive as tools and data grow. A supervisor plus specialized agents gives developers one interface while preserving narrower agent responsibilities. Inline agents replaced deployed stateful agent trees because prompt and tool iteration was too slow through CI/CD-only deployments. MCP support reduces future integration work by turning standardized servers into action groups. Restricted IAM and confirmation gates let the assistant perform operational work without handing the LLM broad destructive authority.

Models used: Amazon Bedrock agents · Amazon Bedrock inline agents · Amazon Bedrock foundation models · Large language models · Amazon DynamoDB working memory · LangGraph · CrewAI

Scale

• Electrolux is described as a company a bit over 100 years old.
  “it's a bit over 100 year old Swedish company headquartered here in Stockholm”transcript, stated
• Electrolux sells about 60 million appliances every year, with an increasing share connected through the cloud.
  “We sell some 60 million every year and ever increasing amount of these appliances are connected smart appliances.”transcript, stated
• AWS frames agent sprawl at scales from five or ten agents to thousands of agents in global organizations.
  “Soon you have to maybe build five or 10 or 50 or 100 or maybe you are a multi-billion organization operating globally.”transcript, stated

  “You have to go to thousands of agents out there.”transcript, stated
• Electrolux found that three knowledge-based agents per data source worked best for their documentation.
  “we come to the conclusion that it works best if we have three different agents per data source”transcript, stated
• The assistant rates SRE-channel answer quality on a one-to-five scale.
  “we allow our developers to rate interactions with this annoying popup where they can rate one to five and add some feedback”transcript, stated

  “here one is like hallucinated and some wrong information. This is obviously not good, but it happens.”transcript, stated
• The team expects Infra Assistant to scale better than humans if support volume jumps by 50x.
  “if we would get like 50 times more questions one day for whatever reason, Infos Assistant would not be bothered by that”transcript, stated
• Complex operations can take up to several minutes.
  “depending on complexity of the task it might take up to several minutes for in persistent to fulfill the like or execute some code”transcript, stated
• Infra Assistant accesses three main sources of organizational information.
  “as it has access to all three main sources of information within our organization”transcript, stated

Alternatives considered (roads not taken)

Keep expanding one large agent with every tool and data source.

Rejected because: The AWS framing says single agents become complex, confused, slower, and more expensive as prompts accumulate tool and data definitions.

Deploy many specialized agents behind many user interfaces.

Rejected because: This creates a splintered user experience and leaves users choosing among many systems; Bedrock multi-agent collaboration instead connects specialized agents behind one interface.

Use a simpler router pattern for all work.

Rejected because: The router pattern is suitable for intent classification, but complex workflows require a supervisor that can plan, delegate, iterate, and aggregate across sub-agents.

Deploy the entire agentic tree as persistent Bedrock state through CI/CD for every prompt or tool change.

Rejected because: As the assistant became more complex, deployments took too long for frequent prompt and agent iterations; inline agents let the service send the current tree as JSON per request.

Write custom tools for every external API integration.

Rejected because: API agents and MCP servers reduce integration effort: OpenAPI plus Lambda lets the agent choose endpoints, and MCP servers can be converted into action groups.

Give the troubleshooting agent broad infrastructure permissions.

Rejected because: The team gave the AWS CLI agent a restricted read-only IAM role so it could diagnose infrastructure without being able to deprovision clusters.

Patterns

multi-agent-orchestration (agent-architecture)

Infra Assistant uses a main supervisor agent to reason over tasks and delegate to specialized knowledge, API, action-group, onboarding, MCP, and troubleshooting agents.

rag-retrieval (retrieval)

The first Infra Assistant MVP was a Bedrock knowledge-based RAG agent over Electrolux documentation and previous chats, and the production system still uses source-specific knowledge-based agents.

tool-use-orchestration (tooling)

Infra Assistant delegates operational work to OpenAPI-backed API agents, Lambda action groups, MCP-derived action groups, and a troubleshooting AWS CLI agent.

human-in-the-loop-gate (safety)

Sensitive Lambda-backed action groups can require user confirmation, returning the planned execution for approval or rejection before the agent proceeds.

observability-tracing (operations)

Bedrock traces expose how the system plans, decomposes tasks, delegates to agents, selects APIs, and aggregates results so the team can debug and improve the multi-agent workflow.

eval-driven-development (quality)

The team rates interactions with developer feedback and a one-to-five SRE-channel accuracy scale, and plans to move from end-to-end evaluation to per-agent evaluation.

safety-policy-enforcement (security)

The AWS CLI troubleshooting agent is constrained by an isolated read-only IAM role, and sensitive actions can be gated by confirmation before execution.

latency-optimization (performance)

The team adopted inline agents partly to reduce iteration and runtime overhead, but latency remains a roadmap item because complex tasks can still take several minutes.

Tradeoffs

Use multi-agent collaboration instead of one large assistant.

Chose A main supervisor coordinating specialized agents over A single agent with every tool, prompt, and data source.

Rationale: Specialization reduces confusion and prompt/tool bloat while allowing one user-facing assistant to handle complex workflows.

Use a supervisor pattern for Infra Assistant's complex operations.

Chose Supervisor-style planning and delegation over Router-only intent classification.

Rationale: The router pattern can pick one specialist, but troubleshooting, onboarding, and multi-step operations need planning, delegation, and aggregation across multiple agents.

Switch from deployed Bedrock agent trees to inline agents.

Chose Runtime-generated JSON agent trees sent with each question over Persistently deployed agent trees updated through CI/CD for each change.

Rationale: Inline agents reduce state-drift concerns and make prompt/tool iteration faster, at the cost of generating the agent tree dynamically at request time.

Use MCP servers as a capability-extension interface.

Chose Convert MCP servers into Bedrock action groups over Writing bespoke tools for each external endpoint or team-owned integration.

Rationale: MCP gives a standardized context/tool interface, making GitHub and other future integrations easier to plug into the agentic tree.

Allow the troubleshooting agent to inspect AWS accounts under read-only permissions.

Chose Arbitrary AWS CLI commands constrained by an isolated restricted role over No account-level CLI access, or broad operational permissions.

Rationale: The agent needs real cloud-state visibility to diagnose issues, but read-only IAM keeps it from making destructive changes.

Use human confirmation on sensitive custom actions.

Chose Approval prompts with the planned action output over Letting the LLM execute sensitive Lambda code without explicit user approval.

Rationale: The team does not fully rely on the LLM for sensitive code paths, so the user can approve or reject the action before it proceeds.

Keep human SRE ownership while letting the assistant handle repetitive work.

Chose Capacity augmentation and context gathering over Replacing SRE team members with the agent.

Rationale: Infra Assistant saves time and scales routine support, but the team explicitly says it is not replacing humans and still needs intervention when knowledge or accuracy is insufficient.

Gotchas (what bit them)

A single agent becomes hard to operate as tools and data accumulate.

Root cause: Tool definitions and data become part of the prompt, which increases complexity, confusion, latency, and cost.

Resolution: Split capabilities into specialized agents and orchestrate them through router or supervisor patterns.

Many specialized agents can fragment the user experience.

Root cause: Users may otherwise need to pick among many separate agent interfaces and know which one can solve their task.

Resolution: Put a router or supervisor in front of specialized agents so developers interact with one assistant.

Stateful agent-tree deployments slowed iteration.

Root cause: Prompt, link, and tool changes had to move through a deployment pipeline that recreated the Bedrock agentic tree.

Resolution: Adopt inline agents so the service can generate and send the current tree JSON with each request.

Operational agents need tight permission boundaries.

Root cause: The troubleshooting agent can run arbitrary AWS CLI commands, so broad credentials would create destructive capability.

Resolution: Run the AWS CLI agent in an isolated environment with a restricted read-only IAM role.

Complex multi-agent operations can be too slow for urgent support.

Root cause: The assistant may need to reason across multiple agents, tools, and code-executing workflows before responding.

Resolution: Treat latency reduction as a roadmap item and avoid using Infra Assistant for everything when developers need immediate help.

End-to-end evaluation hides weak or unused sub-agents.

Root cause: A total Infra Assistant score does not show which individual agent caused a failure or whether an agent is unused.

Resolution: Move toward per-agent evaluation so the team can fix one component or delete unused agents.

Hallucinations still occur despite grounding and tool use.

Root cause: The assistant is an LLM system and is not fully accurate across all questions and actions.

Resolution: Keep references, developer ratings, SRE accuracy scoring, and improvement loops in place rather than treating the agent as fully autonomous.

Evaluation

Electrolux evaluates Infra Assistant through Bedrock traces, source references, developer feedback popups, SRE-channel accuracy ratings, qualitative cost/time comparison, and planned per-agent evaluation. Their current scoring includes hallucination/wrong-answer cases at 1 and fully resolved senior-teammate-level cases at 5.

Metrics: Developer popup rating from 1 to 5 plus written feedback · SRE-channel answer accuracy rated from 1 to 5 · Number of rating-5 interactions that require no human intervention · Latency for complex operations, which can be up to several minutes · Manual time and monthly cost comparison · Future per-agent performance and usage metrics

Outcomes

• Infra Assistant became the first responder in the support channel and can also be used privately.
• It answers Electrolux-specific documentation questions with references to documents or previous chats.
• The onboarding example delegated work to an onboarding supervisor and Atlassian API agent and completed the onboarding task.
• The troubleshooting agent correctly diagnosed a deliberately introduced transit-gateway/security-group issue on the first try.
• The team reports time savings on repetitive tasks and cost efficiency compared with manual effort.
• The system still has limitations around speed, accuracy, and per-agent evaluation.

The case is production-oriented but qualitative: it reports 60 million appliances sold yearly, a small SRE team, three main internal information sources, one-to-five evaluation scales, possible 50x support-load elasticity, and several-minute task latency, but it does not disclose exact support-ticket volume, monthly cost, or production success-rate percentages.