Multi-Agent Orchestration for Enterprise Conversational AI

atlassian · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Enterprise conversational AI

Atlassian needed Rovo Chat to handle diverse Jira, Confluence, search, and workflow tasks without the tool confusion and latency problems of a single-agent RAG design.

Constraints:

A single agent had to juggle many tools across different domains.
Intent classifiers reduced the tool search space but were static and generalized poorly.
Pre-planned graph execution was hard to repair when subagents failed or returned incomplete information.
The production system needed both quality and time-to-first-token improvements.

Architecture

Atlassian evolved Rovo Chat from single-agent RAG to hierarchical multi-agent orchestration. Domain-specialized subagents such as a Jira Agent handle focused subtasks, dynamic reasoning modes support brainstorming, tool Q&A, and multi-step reasoning, and the current hybrid orchestrator uses parallel tool calling rather than a brittle pre-planned DAG.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  hybrid-orchestrator["Hybrid Orchestrator"]
  jira-agent["Jira Agent"]
  confluence-agent["Confluence Agent"]
  search-agent["Search Agent"]
  tool-layer["Tool Layer"]
  hybrid-orchestrator --> jira-agent
  hybrid-orchestrator --> confluence-agent
  hybrid-orchestrator --> search-agent
  hybrid-orchestrator --> tool-layer
  jira-agent --> hybrid-orchestrator
  jira-agent --> tool-layer
  confluence-agent --> hybrid-orchestrator
  confluence-agent --> tool-layer
  search-agent --> hybrid-orchestrator
  search-agent --> tool-layer

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Hybrid Orchestrator`	Routes queries, selects reasoning modes, and coordinates parallel tool calls.	`Jira Agent`, `Confluence Agent`, `Search Agent`, `Tool Layer`
`Jira Agent`	Handles Jira-specific tasks, including custom JQL capabilities.	`Hybrid Orchestrator`, `Tool Layer`
`Confluence Agent`	Handles Confluence knowledge and page workflows.	`Hybrid Orchestrator`, `Tool Layer`
`Search Agent`	Retrieves enterprise knowledge across indexed sources.	`Hybrid Orchestrator`, `Tool Layer`
`Tool Layer`	Provides domain tools such as JQL, Confluence search, people lookup, and URL reading.	—

Data flow

A user query enters Rovo Chat. The orchestrator determines whether it needs direct answering, tool Q&A, brainstorming, or multi-step reasoning; routes domain-specific subtasks to expert subagents; calls tools in parallel where possible; and synthesizes the result for the user.

“Their solution involved decomposing complex queries into subtasks handled by domain-specialized subagents”summary, stated

“using a hybrid orchestrator that leverages parallel tool calling.”summary, stated

Why this architecture

Atlassian specialized agents because one agent with dozens or hundreds of tools suffered from cognitive load and tool misclassification. The hybrid orchestrator replaced graph pre-planning because plans were difficult to reconfigure after subagent failures.

“the cognitive load on the LLM leads to tool misclassification and degraded performance.”summary, stated

“when subagents failed or provided incomplete information, reconfiguring the pre-planned graph proved extremely difficult.”summary, stated

Models used: unspecified Rovo Chat LLMs

Scale

The new system improved quality by 3.49%.
“Results showed a 3.49% quality improvement over their baseline”summary, stated
Time to first token improved 75.96% faster at P10 and 29.5% faster at P50.
“with significant latency reductions particularly at the P10 (75.96% faster) and P50 (29.5% faster) percentiles for time to first token.”summary, stated

Alternatives considered (roads not taken)

Single-agent RAG with all tools exposed

Rejected because: Tool overload caused confusion, misclassification, and degraded performance.

“Atlassian's initial implementation followed a standard RAG pattern with all tools exposed at the top level of a single agent.”summary, stated

Pre-planned graph-based multi-agent orchestration

Rejected because: It was difficult to reconfigure after incomplete or failed subagent outputs.

“Without complete information from just the user query, single-shot planning couldn't reliably predict which subagent outputs would be relevant”summary, stated

Patterns

`multi-agent-orchestration` (architecture)

Rovo decomposes enterprise questions into domain-specialized subagent work.

“Their solution centers on decomposing this complexity through specialized subagents organized hierarchically”summary, stated

`tool-use-orchestration` (execution)

The hybrid orchestrator routes tool calls across Jira, Confluence, search, and other domains.

“with an orchestrator that routes queries to appropriate "experts."”summary, stated

`latency-optimization` (performance)

Parallel tool calling and orchestration reduced time-to-first-token percentiles.

“using a hybrid orchestrator that leverages parallel tool calling.”summary, stated

Tradeoffs

Move from graph-planned orchestration to a hybrid orchestrator

Chose Hybrid orchestration with parallel tool calling over Single-shot DAG planning.

Rationale: The graph was hard to repair when subagent outputs were incomplete.

“when subagents failed or provided incomplete information, reconfiguring the pre-planned graph proved extremely difficult.”summary, stated

Create domain-specialized subagents

Chose Jira, Confluence, search, and other experts over One agent with all tools.

Rationale: Specialization reduces cognitive load and tool misclassification.

“Their solution centers on decomposing this complexity through specialized subagents organized hierarchically”summary, stated

Gotchas (what bit them)

Tool overload degraded a single-agent design.

Root cause: The agent had to choose among many overlapping domain tools.

Resolution: Atlassian moved to specialized subagents under a hierarchical orchestrator.

“When an agent must juggle dozens or hundreds of tools across different domains”summary, stated

Static intent classifiers were not flexible enough.

Root cause: They reduced tool search space but struggled with complex multi-step queries.

Resolution: Rovo adopted dynamic reasoning modes and multi-agent orchestration.

“this approach proved static and struggled with generalizability, particularly for complex multi-step queries.”summary, stated

Evaluation

Atlassian compared against the single-agent baseline using quality and latency metrics.

Metrics: 3.49% quality improvement · 75.96% faster P10 time to first token · 29.5% faster P50 time to first token

“Results showed a 3.49% quality improvement over their baseline, with significant latency reductions”summary, stated

Outcomes

Rovo improved quality over the single-agent baseline.
The hybrid orchestrator reduced latency, especially at P10 and P50.
The design reduced tool confusion by decomposing work into domain experts.

The case reports 3.49% quality improvement, 75.96% faster P10, and 29.5% faster P50 time to first token.

“Results showed a 3.49% quality improvement over their baseline, with significant latency reductions particularly at the P10”summary, stated