Agentic AI Architecture for Meeting Intelligence and Productivity Automation

zoom · Tech · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.2 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Meeting productivity

Zoom wanted AI Companion to turn meeting content into summaries, analysis, scheduling, and agenda preparation at low latency and cost.

Constraints:

Low latency
Cost-effective scale
Meeting context across sessions
Security

Architecture

Zoom AI Companion 3.0 is a modular multi-agent framework for meeting productivity. It handles memory, context, prompts, history, custom templates, workflows, and tenant isolation; uses advanced context engineering and long-term memory to tailor summaries and scheduling to user preferences; pairs Zoom small language models with third-party frontier LLMs through a federated AI design; runs on AWS microservices with Bedrock access; and uses OpenSearch indexes/vector stores to retrieve conversation context without delay. SLMs preprocess intent and structure before LLM calls so outputs are tailored while controlling cost and latency.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  ai-companion-interface[AI Companion interface] --> agentic-framework[Agentic framework]
  agentic-framework --> context-engineering[Context engineering]
  context-engineering --> opensearch-context-store[OpenSearch context store]
  agentic-framework --> federated-ai-router[Federated AI router]
  federated-ai-router --> small-language-models[Small language models]
  small-language-models --> large-language-models[Large language models]
  large-language-models --> bedrock-access[Bedrock access]
  bedrock-access --> aws-microservices[AWS microservices]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`AI Companion interface`	User-facing meeting, scheduling, agenda, and analysis assistant.	`Agentic framework`
`Agentic framework`	Native multi-agent framework for memory, context, prompt, history, templates, workflows, and tenant isolation.	`AI Companion interface`, `Context engineering`, `Federated AI router`
`Context engineering`	Combines session information, long-term memory, and personal preferences.	`Agentic framework`, `OpenSearch context store`
`OpenSearch context store`	Index and vector store for serving conversation context quickly.	`Context engineering`
`Federated AI router`	Routes work between small Zoom models and frontier LLMs.	`Agentic framework`, `Small language models`, `Large language models`
`Small language models`	Preprocess user intent and output structure before larger-model generation.	`Federated AI router`, `Large language models`
`Large language models`	Frontier models used for final generation and reasoning.	`Small language models`, `Bedrock access`
`Bedrock access`	AWS model API layer available to Zoom AI Companion.	`Large language models`, `AWS microservices`
`AWS microservices`	Modular scalable deployment substrate for regions, load, and services.	`Bedrock access`

Data flow

Meeting and user context enters AI Companion. The agentic framework uses memory, contact, prompt, history, templates, and workflow definitions; OpenSearch indexes conversations and retrieves context; small language models infer intent or desired structure; larger LLMs generate the final tailored output; Bedrock and AWS microservices support scalable serving for summaries, scheduling, agenda creation, and cross-meeting analysis.

3 supporting quotes

“It is a native, multi-agent framework that handles memory, contact, prompt, history, et cetera. There are three key principles we build on top.”transcript, stated

“OpenSearch with index, with vector store, serving conversations and retrieving that information when it's required without any delays.”transcript, stated

“The small language models will understand that intent first, process it, and then pass that information to an LLM”transcript, stated

Why this architecture

Zoom chose a federated multi-agent design to balance quality, latency, cost, personalization, and scale. Small models can cheaply infer intent or formatting before larger-model calls, OpenSearch supplies context quickly, tenant-isolated workflows adapt to verticals, and AWS microservices let the service scale across regions and demand patterns.

3 supporting quotes

“We wanna achieve the lowest latency. And also we wanna scale at a cost-effective way. How we really achieved that, first of all, we built AI Company 3.0 on the agentic framework.”transcript, stated

“At Zoom, we use our own small language model paired with frontier larger language model from third parties to provide the highest quality to our customers and also control the cost in a way that we can scale out easily.”transcript, stated

“And for Zoom, serving thousands and millions of customers, scalability is one of the key priorities.”transcript, stated

Models used: small language models · frontier larger language models

Scale

The framework targets thousands and millions of Zoom customers.
“And for Zoom, serving thousands and millions of customers, scalability is one of the key priorities.”transcript, stated
AI Companion can reduce days of coordination to a few minutes in the scheduling example.
“So what usually took me days of coordination, AI Companion does it in a few minutes. So from meeting recap, to agentic AI, AI Companion evolves from reactive to proactive.”transcript, stated

Alternatives considered (roads not taken)

Send every task directly to a large frontier model

Rejected because: Zoom uses small models first to control cost and latency while preserving quality.

Use only session-local meeting context

Rejected because: AI Companion uses long-term memory and personal preference for tailored outputs.

“What we do is when AI Companion doesn't work for you, it take into consideration not just the session information, but also the long-term memory and your personal preference. For example, for me, I prefer short meetings.”transcript, stated

Patterns

`multi-agent-orchestration` (meeting-workflow)

Zoom frames AI Companion as a native multi-agent framework for meeting workflows.

“we built AI Company 3.0 on the agentic framework. It is a native, multi-agent framework that handles memory, contact,”transcript, stated

`latency-optimization` (serving)

The design explicitly optimizes for low latency at scale.

“We wanna provide the highest quality to our customer. We wanna achieve the lowest latency. And also we wanna scale”transcript, stated

Tradeoffs

Federate model usage between small and frontier models.

Chose Small language models paired with frontier LLMs over Sending every meeting-intelligence task directly to the largest model.

Rationale: Zoom needed quality, latency, and cost control at collaboration-platform scale.

How to balance quality, latency, and cost.

Chose federated small-model plus frontier-LLM routing over frontier LLMs for every step.

Rationale: SLMs preprocess intent and structure while LLMs handle heavier generation, supporting scale and cost control.

“SLMs serving and processing that input before it goes to larger models, to LLMs.”transcript, stated

Gotchas (what bit them)

Generic meeting summaries did not create enough personal value for users.

Root cause: A single meeting recap can miss the user's role, preferences, risks, and action-item needs.

Resolution: Zoom used role-aware AI Companion context and personal preferences to tailor summaries and follow-up artifacts.

“Meeting summaries are sometimes general, and as a result, may lack personal value. But with AI Companion Web 3.0, it knows your role and can tailor the summary”transcript, stated

Evaluation

The transcript emphasizes latency and scalable deployment rather than a formal benchmark report.

Metrics: latency · scale

“quality to our customers and also control the cost in a way that we can scale out easily. Next, I give it to Dmitriy”transcript, stated

Outcomes

The summary says meeting administration that took days can be reduced to minutes.

The system supports meeting summaries, cross-meeting analysis, scheduling, and agendas.

“eting analysis, schedule meetings with intelligent calendar management, and prepare meeting agendas—reducing what typically takes days of administrative work to minutes while maintaining low latency and cost-effectiveness at scale.”summary, stated