Progressive Tool Discovery for MCP Servers to Manage Context at Scale

amazon · Media & Entertainment · 2026 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production AI agent infrastructure and MCP tool management

Amazon Prime Video needed centralized MCP tool infrastructure without forcing every agent session to load hundreds of tools into context, because excessive tool exposure degraded tool choice, increased hallucinations, and made shared cross-team servers harder to scale.

Constraints:

• Agents usually needed only a few task-specific tools, while the shared MCP server could expose hundreds.
• The architecture still needed centralized tool governance to avoid every team rebuilding the same tools.
• Agents had to discover tools dynamically during a session, not only through static per-agent configuration.
• The solution depended on MCP client support for tools-list-changed notifications and session identifiers.

Architecture

Prime Video kept the shared MCP server but changed the agent's initial context surface. At initialization the server exposes only a single find-tools capability. Tools are mapped to problem categories, the MCP server tracks the agent session's current category, and a tools-list-changed notification prompts the client to fetch a refreshed category-specific tool list. The same session can switch categories later, replacing one tool subset with another while preserving centralized governance.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  agent-session["Agent session"] --> find-tools-capability["Find tools capability"]
  find-tools-capability["Find tools capability"] --> problem-category-map["Problem category map"]
  find-tools-capability["Find tools capability"] --> tool-registry["Tool registry"]
  find-tools-capability["Find tools capability"] --> mcp-session-tracker["MCP session tracker"]
  problem-category-map["Problem category map"] --> category-specific-tool-subset["Category-specific tool subset"]
  tool-registry["Tool registry"] --> problem-category-map["Problem category map"]
  tool-registry["Tool registry"] --> tool-governance-process["Tool governance process"]
  mcp-session-tracker["MCP session tracker"] --> tools-list-changed-notification["Tools list changed notification"]
  tools-list-changed-notification["Tools list changed notification"] --> mcp-client-refresh["MCP client refresh"]
  mcp-client-refresh["MCP client refresh"] --> category-specific-tool-subset["Category-specific tool subset"]
  category-specific-tool-subset["Category-specific tool subset"] --> agent-session["Agent session"]
  prime-video-production-agents["Prime Video production agents"] --> agent-session["Agent session"]

Components

Data flow

An agent session initializes against the MCP server and sees only the find-tools capability. When the agent needs a category such as operations, it calls find tools with that category. The server records the session-to-problem-space mapping, returns discovered tool names, sends a tools-list-changed notification, and the MCP client refreshes its tool list. The agent then works with the smaller category-specific subset; if the task changes, the agent invokes find tools again and the server swaps the active category and exposed tool set.

Why this architecture

Prime Video chose progressive discovery because it preserved the organizational benefits of a central MCP server while reducing the context cost seen by any one agent session. Static per-agent configuration and sub-agent discovery were considered, but dynamic mid-session discovery better matched tasks that can move between problem spaces. The architecture accepts extra round trips because a smaller, more relevant tool context improved agent performance and tool selection.

Models used: MCP-compatible AI agents · MCP server · Prime Video internal agent clients

Scale

• The central MCP server exposed hundreds of tools.
  “Agents connecting to this centralized MCP server suddenly had access to hundreds of tools in their context window.”summary, stated
• Typical tasks needed only three or four tools.
  “The team observed that agents typically needed only three or four specific tools to accomplish any given task”summary, stated
• The shared-server approach supported one server with many more tools than the team previously considered acceptable.
  “enabling them to operate "one server with many more tools than we would have been okay with before."”summary, stated
• Prime Video agents are used for live event streaming quality analysis, experiment analysis, incident triage, and code changes.
  “Their agents now handle real production responsibilities including real-time quality analysis for live streaming events, production feature experiment analysis, operational incident triage and response, and contributing code changes.”summary, stated

Alternatives considered (roads not taken)

Independent team-specific MCP servers

Rejected because: They duplicated tools across teams and increased maintenance overhead.

Expose all centralized MCP tools at initialization

Rejected because: Hundreds of tools bloated context and degraded tool selection.

Sub-agent tool discovery

Rejected because: It was considered but did not fit the team's scaling needs as well as full dynamic discovery.

Per-agent tool configuration files

Rejected because: Configuration distribution would become the tool-context control plane, while the team needed mid-session discovery.

Patterns

context-engineering (context-management)

Prime Video treated tool definitions as scarce context and exposed only the discovery mechanism until the agent selected a problem category.

tool-use-orchestration (architecture)

The MCP server orchestrates tool availability dynamically through find tools, category mappings, session state, notifications, and refreshed tool lists.

structured-output-contract (interface)

The architecture relies on MCP protocol contracts: session IDs, tools-list-changed notifications, and tool-list refresh behavior.

eval-driven-development (quality)

The team validated the approach by comparing tool-selection performance and hallucination behavior before and after reducing active tools.

safety-policy-enforcement (governance)

Progressive discovery did not replace governance; the shared tool server still needs standards for tool definitions, overlap, and contributions.

Tradeoffs

Keep centralized shared MCP infrastructure

Chose A common remote MCP server with progressive discovery over Independent MCP servers per team.

Rationale: Centralization reduced duplicated tool implementation, while progressive discovery limited the context cost of that centralization.

Use dynamic discovery during the agent session

Chose Find-tools-driven mid-session category switching over Static per-agent tool configurations.

Rationale: The team needed agents to discover relevant tools as problem context changed rather than relying entirely on predistributed configuration.

Pay extra discovery round trips

Chose Progressive discovery latency over Loading every tool once at initialization.

Rationale: For Prime Video's use cases, the reduced context bloat and better tool selection outweighed the added interaction overhead.

Use MCP session tracking for category state

Chose Session ID to problem-space mapping over A fully stateless discovery protocol.

Rationale: Session tracking made it possible to know which category-specific tools should be visible to a given agent session, but created exposure to future MCP protocol changes.

Gotchas (what bit them)

Centralizing tools can move duplication out of teams but move context bloat into the agent.

Root cause: The shared MCP server accumulated hundreds of cross-team tools, while each task needed only a small subset.

Resolution: Expose only find tools initially and load category-specific tools through progressive discovery.

Tool bloat can reappear inside broad categories.

Root cause: Poorly designed categories or overlapping tool definitions can still expose too many tools after discovery.

Resolution: Maintain governance over tool definitions, category design, overlap, and shared-server additions.

Progressive discovery adds interaction latency.

Root cause: The agent must invoke find tools, receive results, handle a notification, and fetch an updated tool list.

Resolution: Accept the extra round trips for workflows where reduced context bloat improves performance enough to justify them.

Protocol non-compliance can break dynamic discovery.

Root cause: Some MCP clients do not respond correctly to tools-list-changed notifications.

Resolution: Prime Video avoided the issue by controlling its internal clients; external deployments need compatibility checks.

Evaluation

Prime Video compared agent behavior before and after reducing active tool context, watching tool-selection quality, performance, and hallucinations. The case reports measurable improvement but does not disclose exact metrics.

Metrics: number of tools exposed per task · performance improvement · hallucination reduction · tool selection reliability · server tool capacity

Outcomes

• Prime Video could operate a shared MCP server with more tools than it would previously have accepted.
• Agents received only three or four context-appropriate tools for a task rather than hundreds.
• The approach improved performance and reduced hallucinations according to the case study, though exact metrics were not disclosed.
• The pattern supported production agent workflows across streaming quality analysis, experiment analysis, incident response, and code changes.

The concrete scale anchors are hundreds of available MCP tools, three or four active tools per task, one initial find-tools capability, and Prime Video production workflows including live-event streaming quality analysis.