Building a Production MCP Server for AI Assistant Integration

hugging_face · Tech · 2024 · source Agentic: no · promotional bias: high · source-backed · extracted with prompt 0.5.0 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Tech

Hugging Face wanted AI assistants to access the Hub and Spaces through MCP while supporting user customization and production deployment constraints.

Constraints:

• MCP protocol revisions and client support were changing quickly
• different users need different tool sets
• remote server transport choices affect connection complexity
• production deployment should scale horizontally

Architecture

Hugging Face’s MCP server exposes the Hub and Spaces to AI assistants through a remotely accessible URL. Users configure available tools on the Hugging Face MCP settings page; anonymous users get a standard Hub/Image Generator set, while authenticated users use HF_TOKEN or OAuth so the server can look up selected tools, chosen Gradio applications, and ZeroGPU quota per request. The open-source server supports STDIO, SSE, and Streamable HTTP variants, but production launches with Streamable HTTP in stateless direct-response mode to keep horizontal scaling simple and avoid long-lived connection management. Observability dashboards help inspect client connections and tool-list behavior.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  ai-assistant-client[AI assistant client] --> remote-mcp-url[Remote MCP URL]
  remote-mcp-url --> streamable-http-transport[Streamable HTTP transport]
  streamable-http-transport --> stateless-direct-response-server[Stateless direct response server]
  stateless-direct-response-server --> auth-and-quota-lookup[Auth and quota lookup]
  auth-and-quota-lookup --> dynamic-tool-configuration[Dynamic tool configuration]
  dynamic-tool-configuration --> hub-and-spaces-tools[Hub and Spaces tools]
  stateless-direct-response-server --> hub-and-spaces-tools
  stateless-direct-response-server --> observability-dashboard[Observability dashboard]
  open-source-server-variants[Open-source server variants] --> streamable-http-transport

Components

Data flow

An AI assistant or MCP client connects to the remote hf.co/mcp URL and runs MCP initialization requests such as Initialize, Notifications/Initialize, tools/list, and prompts/list. For each request, the server uses the supplied HF_TOKEN or OAuth credentials when present, looks up the user’s selected tools and Gradio applications, applies quota rules, and answers with direct request/response semantics. Tool calls can then access Hugging Face Hub resources or Spaces. The open-source server can run other transport modes, but the production path avoids server-maintained session state and relies on clients refreshing connections/tool lists instead of long-lived push notifications.

Why this architecture

The architecture favors low-friction remote access and operational simplicity. Dynamic tools are needed because Hugging Face users want different Hub and Spaces capabilities; a simple URL avoids local installation; Streamable HTTP tracks the current MCP direction; and stateless direct response minimizes resource overhead because the current tool set does not need sampling, elicitation, or server-maintained connection state. The team keeps broader transport support in the open-source server for compatibility while choosing the simpler production profile.

Scale

• MCP protocol revisions were moving quickly, with three revisions in nine months and client support varying by feature.
  “Since its launch in November 2024, MCP has undergone rapid evolution with 3 protocol revisions in 9 months.”article, stated

  “support for different MCP Features and revisions in Client applications varies, providing extra challenges for our design choices.”article, stated
• The production server saw 164 different clients in the first week of July 2025, and about half used mcp-remote as a bridge.
  “The first week of July 2025 saw an astonishing 164 different Clients accessing our Server.”article, stated

  “Approximately half of all Clients use it as a bridge to connect to our remote server.”article, stated
• Control-plane traffic dominates tool usage: Hugging Face observed roughly 100 MCP control messages per tool call.
  “we find there is a ratio of around 100 MCP Control messages for each Tool Call.”article, stated
• Tool-list push notifications would require maintaining as many open connections as active clients, independent of active tool usage.
  “Sending notifications would mean maintaining as many open connections as there were currently active Clients - regardless of active usage”article, stated

Alternatives considered (roads not taken)

STDIO transport for production remote access

Rejected because: STDIO is typically for local client/server use on the same computer, while Hugging Face needed remote access by URL.

"STDIO

Typically used when the MCP Server is running on the same computer as the Client."

— article, stated

HTTP with SSE as the production remote transport

Rejected because: SSE was still used by clients but deprecated in favor of Streamable HTTP, so Hugging Face avoided investing heavily in it.

Stateful server-push streams for production

Rejected because: They support notifications and elicitation but require keep-alive, reconnection/resumption, session affinity or shared state, and many idle client connections.

Static one-size-fits-all tool list

Rejected because: Hugging Face wanted users to customize tools and selected Gradio applications for their own needs.

Patterns

tool-use-orchestration (integration)

The MCP server exposes Hugging Face resources and Spaces as assistant-accessible tools.

observability-tracing (operations)

The implementation includes observability into client connection behavior and tool-list change handling.

Tradeoffs

Use a stateless direct-response production configuration.

Chose stateless direct response over stateful server-push connection management.

Rationale: The summary says this allowed simple horizontal scaling and avoided session-affinity/shared-state complexity.

Which remote MCP transport to use in production.

Chose Streamable HTTP over HTTP with SSE or STDIO as the main production path.

Rationale: Streamable HTTP is the newer flexible remote transport, while SSE was deprecated and STDIO is local-first.

"Streamable HTTP

A more flexible remote HTTP transport that provides more options for deployment than the outgoing SSE version"

— article, stated

Whether to support real-time tool-list change notifications at launch.

Chose client refresh/reconnect behavior instead of server-push notifications over maintaining long-lived connections for every configured client.

Rationale: Immediate push would require many idle connections and notifications could still be missed after inactive clients disconnect.

How to make hf.co/mcp friendly in browsers without breaking clients.

Chose browser-aware friendly instructions over always returning a web page or unfriendly JSON/405 behavior.

Rationale: Friendly instructions improved UX, but returning a page to VS Code caused high polling traffic, so browser detection was needed.

Gotchas (what bit them)

Rapid MCP protocol evolution complicated production design choices.

Root cause: Client applications vary in support for protocol features and revisions.

Resolution: Hugging Face built support across transport variants while choosing a simpler production configuration.

Evaluation

No formal evaluation method is described; observability is used to inspect connection behavior.

Outcomes

The case emphasizes production design choices rather than quantified usage outcomes.